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SIMMARY 


Objective 

The  objective?,  are  (a)  to  clarify  the  meaning  of  organizational  |>ro(lnclivil\  as  it  applies  to  Air  Force 
organizations,  (b)  to  describe  and  critique  prodnelivilv  incasiireincnl  methods.  («■)  to  describe  a  procedure 
lor  generating  measures  uf  productivity  in  Air  Force  organizations,  and  (d)  to  summarize  research  which 
evaluates  productivity  enhancement  methods. 

Background 

Ihiriug  recent  years,  concern  about  organizational  productivity  has  increased  in  the  Department  of 
Defense  (DoD)  and  in  the  Air  Force.  One  of  the  primary  reasons  for  this  concern  is  the  continual  pressure 
on  lite  military  services  to  justify  budget  requests  in  terms  of  outputs.  In  response  to  initiatives  from  the 
Vi  bile  House  to  improve  Federal  government  prodnelivilv.  as  well  as  from  the  DoD.  the  Air  Force 
finalized  its  productiv  ily  improvement  plan  on  to  November  l<)7().  This  plan  directs  all  major  commands 
and  operating  agencies  to  develop  their  own  prodnelivilv  plan,  to  appoint  a  "productivity  principal"  as 
the  contact  person  for  all  productivity  matters,  and  to  report  productiv itv  results  annually  to  Vir  Force 
headquarters. 

To  fulfill  their  responsibilities  under  this  plan,  prodnelivilv  principals  and  operating  managers  must 
understand  the  meaning  of  prodnelivilv  in  an  Air  Force  environment  and  he  aware  of  alternative 
approaches  to  prodnelivilv  enhancement. 

Approach 

Information  was  obtained  from  a  comprehensive  literature  review  and  from  field  visits  to  more  than 
oil  \ir  Force.  \rmy.  Navv.  and  civilian  organizations  engaged  in  productivity  improvement  efforts. 
Results  were  analyzed  and  used  to  develop  a  methodology  for  generating  criteria  for  measuring 
organizational  prodnelivilv.  Vddilional  information  was  obtained  from  a  simulated  field  test  of  the 
proposed  productivity  measurement  melhodologv  conducted  at  an  Air  Force  Avionics  Maintenance 
Squadron. 

Specifics 

The  report  ha-  -even  chapters  and  an  appendix.  Chapter  I  is  an  introduction.  Chapter  II  rev iew s 
varion-  definition'  of  prodnelivilv  and  contrasts  them  with  two  related  concept; — quality  ol 
working  life  anti  organizational  effectiveness.  Chapter  III  reviews  the  definition  ol  productivity  a?  it 
relate-  'pccilicallv  to  the  \ i r  Force.  Chapter  l\  provides  a  taxonomy  and  critique  ol  measures  ol 
efficiency.  effect iv cue",  and  combined  methods  that  integrate  clliciency  ami  elleclivetless  measures 
into  a  enmprehen-ive  measurement  strategy.  Chapter  \  draws  conclusions  Irom  the  review  and 
implication'  for  \ir  Force  prodnelivilv  measurement  methods,  (.hapter  \l  describes  a  method  lor 
generating  prodnelivilv  criteria  for  Vir  Force  organizations  ami  Chapter  Nil  describes  a  Held 

den . .  ration  ol  the  met  hodologv .  Since  the  ultimate  purpose  ol  measuring  product  iv  it  y  is  to  make 

improvement',  the  \ppcmliv  highlight'  the  major  findings  of  the  literature  on  productivity 
enhancement. 

Organizational  prodnelivilv  in  the  Vir  Force  has  recenllv  emerged  as  an  important  hut  not  vet  well- 
defined  area  of  concern.  Rased  on  Vir  force  requirements,  a  procedure  was  described  lor  generating 
measure'  ol  productiv  if «  in  Vir  Force  organizations.  Based  oil  a  simulated  Held  lest,  it  appeared  that  the 
melhodologv  oiler'  'itllicicul  promise  to  juslilv  the  recommendation  that  it  he  iurthcr  tested  in  a  number 
of  functional  Vir  Force  organizations. 
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PREFACE 


This  report  is  a  third  interim  technical  report  for  Contract  No. 
F-33615-79-C-0019 ,  Taxonomy  and  Codification  of  Productivity  Criteria. 

This  18-month  effort  has  focused  on  reviewing  and  critiquing  existing  ap¬ 
proaches  to  defining  and  measuring  productivity  and  drawing  implications 
from  existing  methods  for  use  in  measuring  productivity.  This  research 
was  completed  under  Work  Unit  7734-08-10.  Two  previous  interim  reports 
were  produced  under  this  contract:  Manager's  Guide  to  Productivity  Im- 
provement  Resources  and  Programs,  TP  81-12,  and  Measuring  and  Enhancing 
Organizational  Productivity:  An  Annotated  Bibliography,  TR-81-6. 

This  research  is  part  of  a  larger  effort  to  develop  methodologies 
for  assessing  and  improving  productivity  and  performance  in  Air  Force  or¬ 
ganizations.  In  the  current  environment  where  a  renewed  awareness  of  the 
need  for  effective  military  organizations  is  coupled  with  strong  fiscal 
pressures,  the  level  of  concern  for  productivity  has  increased  at  all 
levels.  This  research  area  not  only  addresses  an  important  area  of  re¬ 
search,  but  also  an  urgent  national  need. 

The  author  wishes  to  thank  Ms.  Elizabeth  C.  Clark  for  her  able  admin¬ 
istrative,  editorial  and  clerical  assistance,  Mr.  Robert  E.  Wilkinson 
who  provided  valuable  input  to  the  development  of  the  productivity  mea¬ 
surement  method,  and  Ms.  Myra  Palla  who  typed  the  final  manuscript. 
Appreciation  is  also  expressed  to  the  AFHRL  project  monitor,  Dr.  Charles  N. 
Weaver,  for  his  valuable  comments  and  suggestions.  Finally  the  author 
wishes  to  thank  the  previous  AFHRL  project  monitors  who  provided  signif¬ 
icant  assistance  in  earlier  phases  of  the  research:  Major  John  0. 

Edwards,  Jr.,  Dr.  William  E.  Alley,  Dr.  Joe  T.  Hazel,  and  Mr.  William  L. 
Titsworth. 


ii 


TABLE  OF  CONTENTS 


Page 

I.  INTRODUCTION  .  5 

II.  WHAT  IS  PRODUCTIVITY? .  7 

Productivity — The  Economist's  View  .  7 

Productivity — The  Engineer's  View  .  8 

Productivity — The  Accountant's  View  .  9 

Productivity — The  Manager's  View  .  10 

Productivity — The  Industrial/ Organizational 

Psychologist's  View  .  11 

Organizational  Effectiveness  .  12 

Effectiveness  and  Productivity  .  15 

Quality  of  Working  Life  .  17 

Quality  of  Working  Life  and  Productivity  .  19 

Summary  .  20 

III.  PRODUCTIVITY— AN  AIR  FORCE  VIEW  .  21 

Productivity — The  Official  Air  Force  View  .  21 

Air  Force  Productivity — A  View  From  the  Field  .  23 

Air  Force  Productivity — A  Researcher's  View  .  25 

Summary  of  Field  View  of  Productivity  .  28 

Productivity  in  Air  Force  Organizations: 

A  Proposed  Conceptualization  .  29 

IV.  PRODUCTIVITY  MEASUREMENT  METHODS:  DISCUSSION  AND  CRITIQUE. . .  31 

General  Measurement  Issues  .  31 

Characteristics  of  Good  Productivity  Measures  .  33 

Review  and  Critique  of  Productivity  Measures  .  35 

Efficiency  Measurement  .  37 

Effectiveness  Measurement  .  52 

Integrated  Approaches  .  61 

Impact  of  Measurement  on  Productivity  .  73 

V.  CONCLUSIONS  AND  IMPLICATIONS  FOR  AIR  FORCE  PRODUCTIVITY 

MEASUREMENT  .  76 

Conclusions  From  Previous  Research  .  7® 

Implication  for  Methodology  Development  .  80 

VI.  METHODOLOGY  FOR  GENERATING  PRODUCTIVITY  INDICATORS  .  83 

Overview  of  the  Measurement  Procedure  .  83 

Description  of  the  Measurement  Procedure  .  84 

Phase  1.  Background  Phase  .  84 

Phase  2.  Definition  of  Key  Results  Areas  .  86 

Phase  3.  Definition  of  Productivity  Indices  and 

Measurement  Methods  .  80 


1 


Table  of  Contents  (Continued) 


Page 

Phase  4.  Generalization  to  Other  Units  (Optional)  .  93 

Phase  5.  Implementation  .  96 

VII.  APPLICATION  OF  THE  METHODOLOGY:  AN  EXAMPLE  .  97 

Phase  1.  Background  .  97 

Phase  2.  Definition  of  Key  Results  Areas  .  98 

Phase  3.  Definition  of  Productivity  Indices  and 

Data  Sources  . 100 

Phase  4.  Generalization  of  Indicators  . 107 

Phase  5.  Implementation  . 107 

Summary  and  Critique  . 112 

REFERENCES  . 113 

APPENDIX  A:  Programs  to  Enhance  Productivity:  Review  of 

Research  Results  .  119 


LIST  OF  FIGURES 

Figure  Page 

1  Elements  of  the  Production  Process  .  29 

2  Taxonomy  of  Productivity  Measurement  Methods  .  36 

3  Types  of  Efficiency  Ratios  .  37 

4  Taxonomy  of  Efficiency  Measures  .  38 

5  Illustration  of  Performance  Efficiency  Computation  .  46 

6  Process  of  Deriving  Implications  from  Conclusions  .  80 

7  Guide  to  Determining  Group  Membership  .  88 

8  Organizational  Diagram — Communications  Navigation  Branch  .  99 


LIST  OF  TABLES 


TABLE  Page 

1  Manager's  Definition  of  Productivity  .  11 

2  Computation  of  Labor  Partial  Index  .  41 

3  Procurement  Actions,  Complexity  Factors  and 

Time  Estimates  .  50 

4  Labor  Efficiency  Indicies  .  64 

5  Comparison  of  Labor  Efficiency  and  Total  Cost  Per  Unit  of  Output  64 

6  Behavioral  Definitions  and  Computational  Formulas  .  68 

7  Criterion  Categories  to  Consider  in  Developing  a  Set  of 

Productivity  Measures  for  an  Air  Force  Organization  .  77 


2 


1 


List  of  Tables  (Continued) 


Table  Page 

8  Guidelines  for  Selecting  Size  of  Measurement  Sample  .  94 

9  Key  Results  Areas  From  Group  A  . 101 

10  Group  B  Results  . 104 

11  Final  Key  Results  Areas,  Indicators  and  Data  Sources  . 108 

A-l  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Selection  and  Placement  Programs  . 121 

A-2  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Training  and  Instruction  Programs  . 122 

A-3  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Appraisal  and  Feedback  Programs  . 123 

A-4  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Management  by  Objectives  . 124 

A-5  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Goal  Setting  . 124 

A-6  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Financial  Compensation  Programs  (All  Types  Combined)  . 125 

A- 7  Number  and  Percentage  of  Studies  Showing  Productivity  Improvement 

From  Financial  Compensation  by  Program  Type  . 125 

A-8  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Job  Design  Programs  . 126 

A-9  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Group  Design  Programs  . 127 

A-10  Number  and  Percentage  of  Studies  Showing  Improvement  From  Changes 

in  Supervisory  Methods  . 128 

A-ll  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Changes  in  Organizational  Structure  . 128 

A-12  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Changes  in  Work  Schedule  . 129 

A-13  Number  and  Percentage  of  Studies  Showing  Improvement  From 

Changes  in  Socio-Technical  Systems  . 130 


i 


1 


3 


I .  INTRODUCTION 


Productivity  in  the  United  States,  as  measured  and  reported  by  the 
Bureau  of  Labor  Statistics,  is  declining  at  an  alarming  rate.  If  present 
trends  continue,  the  United  States  will  fall  behind  our  trading  partners 
Japan,  West  Germany,  Canada,  and  France  in  terms  of  total  productivity  be¬ 
fore  the  end  of  the  1980s  (American  Productivity  Center,  1979) ,  While 
output  per  labor  hour  is  declining,  over  one  third  of  U.S.  workers  recent¬ 
ly  surveyed  claim  that  their  skills  and  abilities  are  not  being  adequate¬ 
ly  utilized  by  their  jobs  (Quinn  &  Staines,  1979) .  The  contrast  between 
declining  productivity  and  the  unused  human  potential  raises  questions 
about  the  ways  in  which  organizations  are  structured  ana  managed. 

This  report  is  intended  to  be  of  assistance  to  researchers  and  mana¬ 
gers  who  are  working  to  improve  the  productivity  of  Air  Force  organiza¬ 
tions.  Before  it  is  possible  to  make  major  improvements,  the  concept  of 
productivity  must  be  understood,  and  methods  must  be  developed  for  mea¬ 
suring  it.  This  report  has  four  objectives.  First,  it  seeks  to  clarify 
the  meaning  of  productivity  as  it  applies  to  Air  Force  organizations. 
Secondly,  it  seeks  to  highlight  promising  measurement  techniques  by  as¬ 
sessing  the  strengths  and  weaknesses  of  existing  approaches  to  organiza¬ 
tional  productivity  measurement.  Third,  as  an  aid  to  productivity 
enhancement,  the  report  summarizes  the  results  of  research  on  those  vari¬ 
ables  which  are  assumed  to  have  a  positive  impact  on  organizational  pro¬ 
ductivity.  Finally,  the  report  integrates  some  of  the  promising  measure¬ 
ment  approaches  into  a  methodology  that  is  appropriate  for  generating 
productivity  criteria  in  Air  Force  organizations. 

The  report  has  seven  chapters  and  an  Appendix.  Chapter  II  reviews 
definitions  of  productivity  and  contrasts  them  with  definitions  of  two 
related  concepts — organizational  effectiveness  and  quality  of  working 
life.  Chapter  III  addresses  the  meaning  of  productivity  in  a  military 
environment — such  as  the  U.S.  Air  Force.  In  Chapter  IV,  various  ap¬ 
proaches  to  measuring  productivity  are  described  and  evaluated.  Chapter 
V  draws  conclusions  from  the  review  and  implications  for  Air  Force 
productivity  measurement  methods.  Chapter  VI  describes  a  method  for 
generating  productivity  criteria  for  Air  Force  organizations  and  Chapter 
VII  describes  a  field  demonstration  of  the  methodology.  Since  the  ultimate 
purpose  of  measuring  productivity  is  to  make  improvements,  the  Appendix 
highlights  the  major  findings  of  the  literature  on  productivity  enhance¬ 
ment. 
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Before  measuring  any  construct,  it  first  must  be  defined.  Defining 
productivity  is  particularly  challenging  because  the  term  has  been  used 
in  many  different  ways.  This  chapter  will  discuss  some  of  the  most  com¬ 
mon  definitions  of  productivity  and  help  clarify  the  concept  by  contrast¬ 
ing  it  with  two  related  concepts--organizational  effectiveness  and  quali¬ 
ty  of  working  life. 


Productivity  -  The  Economist's  View 

According  to  a  leading  productivity  economist,  productivity  is  the 
relationship  between  output  and  its  associated  inputs  when  the  output 
and  inputs  are  expressed  in  real  (physical  volume)  terms  (Kendrick,  1977) . 
This  definition  can  be  depicted  as  a  ratio  expressed  as: 

Productivity  = 

I nputs 

This  definition  of  productivity  is  developed  from  production  theory  which 
attempts  to  explain  the  process  by  which  inputs  to  a  firm  are  transformed 
into  products  to  be  sold  in  the  marketplace.  The  basic  production  func¬ 
tion  states  that  the  volume  of  output  (Q)  is  a  function  (F)  of  the  volume 
of  the  basic  factors  of  production.  These  are  labor  (L) ,  capital  (K) , 
intermediate  products  purchased  from  other  firms  (X)  and  the  level  of 
productive  efficiency  which  changes  over  time  (T) .  This  production 
function  is  expressed  as: 

Q  =  F (L,  K,  X,  T) . 

From  this  function  are  derived  various  productivity  indices.  For  example, 
labor  productivity  =  i  relates  the  volume  of  output  to  the  quantity  of  la¬ 
bor  input  normally  expressed  as  numbers  of  hours  worked.  Capital  produc¬ 
tivity  would  be  2 ,  or  output  per  unit  of  capital  input.  Both  capital  pro¬ 
ductivity  and  labor  productivity  are  referred  to  as  "partial"  productivity 
ratios  by  the  economist,  since  they  relate  output  to  only  one  type  of  in¬ 
put.  If  all  inputs,  labor,  capital,  materials,  energy,  etc.,  are  included 
in  the  denominator  of  the  ratio  we  have  what  is  called  a  "multi  factor"  or 
"total  factor"  productivity  ratio. 

While  economists  typically  define  productivity  as  a  ratio  of  outputs 
to  inputs  stated  in  real  or  physical  terms,  they  sometimes  substitute 
prices  or  costs  for  physical  volumes.  Siegel  (1980)  points  out  that  pro¬ 
ductivity  defined  as  a  "family  of  ratios  of  (a)  input  price  to  (b)  output 
price"  (p.  24)  is  equivalent  to  the  definition  expresssed  in  physical 
quantities. 

When  price  or  cost  formulations  are  used  in  the  numerator  in  place  of 
physical  quantities,  gross  output  values  can  be  misleading.  For  instance, 
the  sales  value  of  output  includes  not  only  the  "value  added"  by  the  pro¬ 
cessing  firm  but  also  the  profits  of  firms  which  supplied  raw  materials, 
supplies,  or  services  that  were  inputs  to  the  production  process.  There¬ 
fore,  the  economist's  definition  of  productivity  is  often  modified  to 
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correct  for  this  problem.  An  example  of  a  value  added  definition  of 
productivity  is: 


Net  output  per  employee 


Value  added  per  year 
Total  number  of  employees 


Here  value  added  represents  the  value  added  to  materials  in  the  process 
of  production  (Norman  &  Bahiri,  1972) . 


In  the  past,  economists  have  been  concerned  primarily  with  macro  in¬ 
dicators  of  productivity.  That  is,  the  indicators  deal  with  whole  econ¬ 
omies  or  with  whole  industries  rather  than  with  single  plants  or  organi¬ 
zations  within  an  economy  or  industry.  As  a  result  of  this  focus,  econ¬ 
omists  are  concerned  with  collecting  data  that  can  be  aggregated  from  the 
"bottom  up,"  i.e.,  from  plant  level  to  industry  level  to  the  level  of  the 
whole  economy.  The  need  to  aggregate  data  and  the  measurement  difficul¬ 
ties  associated  with  aggregation  have  led  economists  to  focus  on  labor 
productivity  indices  to  the  virtual  exclusion  of  other  "partials"  such  as 
capital  productivity  or  total  productivity.  There  is  evidence  that,  in 
the  future,  economists  will  begin  to  focus  more  on  multifactor  productiv¬ 
ity  indices  and  plant  level  data  that  will  permit  inter-firm  comparisons 
(Panel  to  Review  Productivity  Statistics,  1979) . 


Productivity  -  The  Engineer's  View 
While  accepting  the  basic  notion  of  productivity  as  the  ratio  of 
output  produced  per  unit  of  input,  the  engineer's  definition  of  productiv¬ 
ity  differs  from  that  of  the  economist.  Conceptually,  the  engineering 
approach  to  productivity  grows  out  of  the  "normal  engineering  expression 
for  the  efficiency  of  a  machine"  (Norman  &  Bahiri,  1972,  p.  27). 


Efficiency 


Useful  work 
Energy 


1 


Thus,  since  output  (useful  work)  is  a  result  of  input  (energy  supplied)  in 
the  physical  sense,  the  engineer's  ratio  cannot  be  greater  than  unity. 

The  extent  to  which  the  "actual"  output  approaches  the  "potential"  output 
is  a  measure  of  the  efficiency  of  the  process.  While  unity  is  the  upper 
limit  of  efficiency  in  the  physical  sense,  in  financial  ratios  the  value 
can  exceed  1  and,  in  fact,  must  if  the  organization  is  to  be  profitable. 


Considering  productivity  as  basically  synonymous  with  efficiency,  the 
engineering  approach  leads  to  three  definitions  of  productivity  (Norman  & 
Bahiri ,  1972) : 

.  ..  c  _  .  ,  ( Useful  output'' 

1.  Generation  of  useful  output  from  input  1  - - - — — 


f-'  i-  V-'Vi  UU  U  1,  V  i  y  - - 

(  Useful  output! 
\  Input  J 


2.  Efficiency  of  input  utilization 

3.  Output  efficiency 


(  Effective  inputs 
V  Actual  input  J 

G  Actual  output^ 
otential  outpuy 


Productivity  (efficiency)  is  improved  when  more  useful  output  is  produced 
with  respect  to  the  level  of  input  to  the  process.  The  second  statement 
covers  the  efficient  utilization  of  resources  in  the  production  process 
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or  the  degree  to  which  energy,  materials,  capital,  or  hours  of  labor  are 
"useful."  The  third  definition  considers  the  extent  to  which  obtained 
outputs  correspond  to  some  standard  of  the  potential  output. 

The  third  definition  has  stimulated  considerable  activity  in  the 
field  of  industrial  engineering.  Much  of  this  work  concentrates  on  the 
development  of  procedures  for  estimating  potential  output  standards.  The 
practice  of  work  measurement  and  methods-time-measurement  attempts  to  an¬ 
swer  two  basic  questions  (Norman  &  Bahiri,  1972): 

1.  What  is  the  "best"  method  to  do  a  particular  job? 

2.  When  this  best  method  is  used,  what  is  the  standard 
level  of  output  to  be  expected,  given  the  production 
environment,  materials,  labor  force,  etc.? 

Considerable  progress  has  been  made  over  the  years  in  applying  work 
measurement  techniques  to  a  variety  of  workplaces.  Even  when  "engineered" 
standards  are  not  feasible,  other  methods  have  been  developed  to  define 
potential  outputs.  Such  methods  include  use  of  past  performance  history 
or  technical  estimates  made  by  knowledgeable  individuals  (Bryant,  Shall- 
man,  &  Brewer,  1973) ,  or  work  sampling  (Udler,  1978) . 

Another  point  should  be  made  concerning  the  engineer's  definition  of 
productivity  as  the  generation  of  useful  output  by  input  in  statement  num¬ 
ber  1  above.  The  numerator  of  this  expression,  useful  output,  recognizes 
that  not  all  output  from  a  production  process  is  useful.  This  may  be  due 
to  any  number  of  reasons.  The  output  may  not  meet  quality  specifications, 
it  may  have  been  produced  after  the  need  for  the  product  has  passed,  or  it 
may  not  be  consistent  with  the  goals  of  the  organization.  In  any  event, 
the  definition  implies  that  there  are  standards  of  useful  output  which 
are  external  to  the  production  process  itself.  Thus,  productivity  is  not 
defined  as  the  ratio  of  total  output  to  input,  but  as  the  ratio  of  useful 
output  to  input.  This  is  an  important  distinction. 

In  contrast  to  the  economist's  definition  of  productivity,  the  en¬ 
gineering  approach  differs  in  purpose.  Industrial  and  manufacturing  en¬ 
gineers  typically  work  at  the  firm  level,  designing  and  implementing  work 
processes.  As  a  result,  the  engineer's  conception  of  productivity  and 
his  techniques  typically  reflect  this  micro  perspective.  Unlike  the  econ¬ 
omist  who  typically  focuses  on  an  industry  or  whole  economy,  engineers  are 
usually  concerned  with  the  efficiency  of  an  individual,  a  work  group,  or  a 
manufacturing  process.  The  economist's  approach  to  defining  productivity 
is  reflected  in  statistics  developed  for  total  industries  which  are  re¬ 
ported  by  trade  associations,  government  agencies,  banks  and  other  insti¬ 
tutions.  In  contrast,  the  analysis  and  measurement  of  individual  produc¬ 
tion  operations  within  an  organization  are  the  usual  focus  of  the  engi¬ 
neer’s  approach  to  productivity  (A.T.  Kearney,  Inc.,  1978). 

Productivity  -  The  Accountant's  View 

Accountants  concern  themselves  with  the  financial  performance  of  or¬ 
ganizations.  The  "tools"  for  monitoring  financial  performance  in  organi- 


zations  are  financial  ratios.  Theoretically,  there  are  no  limits  to  the 
number  of  financial  ratios  that  could  be  derived.  In  practice,  however, 
only  those  that  are  useful  to  management  should  be  computed  (Norman  & 
Bahiri,  1972) .  Productivity  per  se  has  not  been  of  central  concern  to 
most  accountants  although  this  may  be  changing  (Carr,  1973;  Price  Water- 
house  &  Co.,  1980).  However,  the  use  of  financial  ratios  has  been  of  con¬ 
cern  to  accountants  since  the  early  1920s  (Bliss,  1924) . 

While  they  vary  considerably,  most  financial  ratios  are  broadly  con- 


cerned  with  the  ratio  of  "sales  return  on  capital  employed"  or  on  a  ratio 
of  "profit  to  assets."  Such  measures  are  sometimes  termed  "business  ef¬ 
ficiency"  (Norman  &  Bahiri,  1972).  Many  of  these  ratios  resemble  output/ 
input  productivity  ratios.  Consider  the  following  seven  ratios  for  finan¬ 
cial  control  in  a  decentralized  firm  (Norman  &  Bahiri,  1977)  i 

1. 

Profit/Capital  Employed 

2. 

Profit/Sales 

3. 

Sales/Capital  Employed 

4. 

Sales/Fixed  Assets 

5. 

Sales/Stocks 

6. 

Sales/Employee 

7. 

Profit /Employee 

Since  these  are  sales-based 

ratios  they  have  less  to  do  with  production 

efficiency  and  more  to  do  with  factors  in  the  marketplace.  Therefore, 
they  may  be  misleading  if  interpreted  as  productivity  measures.  Unfor¬ 
tunately,  for  many  decision  makers,  financial  data  provide  the  primary 
source  of  input  on  which  decisions  are  based.  This  reliance  on  financial 
data  points  out  the  need  to  more  fully  integrate  "productivity"  concepts 
into  cost  accounting  systems.  This,  in  fact,  is  happening  through  im¬ 
provements  in  cost  accounting  and  the  development  of  productivity  costing 
procedures  (Norman  &  Bahiri,  1972). 


Productivity  —  The  Manager's  View 

American  managers  have  a  broad  view  of  the  meaning  of  productivity 
(Katzell ,  Yankelovitch ,  Fein,  Ornati,  S  Nash,  1975).  These  authors 
surveyed  two  groups  of  managers — Chief  Executive  Officers  (CEOs)  and 
Industrial  Relations  Officers  (IROs) .  Approximately  2,450  questionnaires 
were  mailed  to  CEOs  and  950  to  IROs.  The  563  completed  questionnaires 
that  were  returned  were  approximately  equally  spread  between  the  two 
groups.  With  only  a  16%  return,  there  is  no  assurance  that  the  sample 
was  representative  of  the  total  population  of  managers.  However,  there 
was  considerable  diversity  among  respondents  in  terms  of  age,  size  of 
firm,  and  geographical  location.  Roughly  half  of  the  respondents  came 
from  manufacturing  organizations. 

In  the  survey,  managers  were  asked  to  indicate  their  agreement  or 
disagreement  with  a  number  of  possible  statements  concerning  the  mean¬ 
ing  of  productivity.  The  results  are  summarized  in  Table  1. 
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Table  1 


Manager's  Definition  of  Productivity3 


Productivity  definition  includes: 

%  Agreeing 

1. 

Quality  as  well  as  quantity 

95 

2. 

Output  per  manhour  in  one 
company  or  organization 

90 

3. 

Overall  efficiency  and 
effectiveness  of  the 
operation 

88 

4. 

Disruptions,  "shrinkage," 
sabotage  and  other  troubles 
even  if  they  are  difficult 
to  measure 

73 

5. 

Rate  of  absenteeism  and 

turnover  as  well  as  output 

70 

6* 

Customer  or  client  satisfaction 

64 

7. 

Employee  loyalty,  morale,  or 
job  satisfaction 

55 

8. 

Ratio  of  output  to  input  by 
industry  or  sector  of  the 
economy,  but  not  by  individual 
organization 

22 

aAdapted  from  Katzell  and  Yankelovitch ,  1975 


Based  on  these  responses,  it  appears  that  most  of  the  managers  would 
find  either  the  economist's  or  engineer's  definition  of  productivity  to  be 
too  narrow.  Virtually  9  out  of  10  managers  would  include  quality,  effec¬ 
tiveness  and  efficiency  in  their  definition;  7  out  of  10  would  add  the 
ideas  of  work  stoppages,  waste,  "shrinkage,"  sabotage,  absenteeism  and 
turnover;  and  6  out  of  10  managers  would  include  measures  of  customer  or 
client  satisfaction.  Such  results  cannot  be  considered  conclusive  evi¬ 
dence  of  the  meaning  of  productivity  to  managers.  However,  the  data  il¬ 
lustrate  that  a  large  group  of  management  policy  makers  consider  produc¬ 
tivity  to  be  a  very  broad  concept. 

Productivity  -  The  Industrial/Organizational  Psychologist's  View 

The  concern  of  industrial/organizational  psychologists  is  with  inves¬ 
tigating  human  behavior  in  organizations.  Productivity,  defined  as  out¬ 
put/input,  is  a  "results"  oriented  variable  that  is  partially  a  function 
of  behavior,  but  is  also  affected  by  other  extraneous  (to  the  psycholo¬ 
gist)  aspects  of  the  work  environment.  Thus,  as  a  criterion  against  which 
to  judge  the  impact  of  various  attempts  to  modify  human  behavior  in  organ- 
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izations,  productivity  has  not  proven  as  useful  as  criteria  which  are  de¬ 
fined  in  terms  of  worker  behavior.  Measures  of  production  quantity  are 
useful  criteria  only  if  the  worker  has  a  direct  influence  over  the  amount 
produced  (Zedeck  &  Blood,  1974) . 

Although  one  often  finds  the  term  productivity  appearing  in  the  lit¬ 
erature  of  industrial/organizational  psychology,  its  meaning  is  usually 
vague.  For  example,  a  classic  theoretical  treatment  of  work  motivation 
contains  the  following  paragraph  which  erroneously  equates  productivity 
and  performance. 

It  is  typically  assumed  by  most  people  connected 
with  the  human  relations  movement  that  job  satis¬ 
faction  was  positively  related  to  job  performance. 

In  fact,  human  relations  might  be  described  as  an 
attempt  to  increase  productivity  by  satisfying 
the  needs  of  employees  (Vroom,  1964,  p.  181) . 

Katz  and  Kahn  in  their  classic  description  of  a  systems  view  of  or¬ 
ganizations  define  productivity  as  "a  measure  of  role  performance"  (1966, 
p.  374) .  These  authors  viewed  productivity  as  a  measure  of  the  output  of 
an  individual  in  his  or  her  work  role.  While  it  was  frequently  broadened 
to  include  work  quality  as  well  as  quantity,  individual  performance  has 
generally  been  the  psychologist's  focus. 

Psychologists  and  other  organizational  researchers  have  typically 
given  more  attention  to  "organizational  effectiveness"  and  more  recently 
"quality  of  work  life"  than  to  productivity.  While  these  concepts  are 
related  to  productivity,  the  nature  of  this  relationship  is  often  implicit 
and  confused.  The  next  two  sections  will  discuss  the  concepts  of  organi¬ 
zational  effectiveness  and  quality  of  working  life  and  will  attempt  to 
clarify  their  presumed  relationships  to  productivity. 

Organizational  Effectiveness 

In  discussing  the  concept  of  organizational  effectiveness,  Mahoney 
and  Weitzel  (1969)  point  out  the  similarities  between  the  problem  of  de¬ 
fining  effectiveness  and  the  "criterion  problem"  that  has  plagued  person¬ 
nel  psychologists  for  many  years.  Thorndike  (1949)  differentiated  three 
sets  of  criteria  for  research  on  employee  selection:  ultimate,  interme¬ 
diate,  and  immediate.  The  ultimate  criterion  is  a  final  goal  stated  in 
general  terms  and  is  not  susceptible  to  assessment  by  outside  observers. 
The  final  criterion  rests  on  the  judgements  of  officials  best  qualified 
to  determine  if  stated  organizational  goals  have  been  achieved.  In  prac¬ 
tice,  various  mid-range  criteria  (immediate  or  intermediate)  are  used  as 
surrogates  to  assess  the  effectiveness  of  organizational  units.  Rational 
processes  are  used  to  determine  the  relevance  of  immediate  and  intermedi¬ 
ate  criteria  to  ultimate  criteria  since  measures  of  ultimate  criteria  ore 
usually  not  available. 

As  various  models  of  effectiveness  are  proposed,  the  number  of  mid¬ 
range  criteria  begins  to  mushroom.  This,  in  fact,  is  what  has  happened  in 
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the  organizational  effectiveness  research  literature.  A  number  of  re¬ 
searchers  have  attempted  to  describe  various  models  of  organizational 
effectiveness  (Mahoney  £  Weitzel,  1969;  Campbell,  Bownas,  Peterson,  & 
Dunnett,  1974;  Engel,  1977;  Goodman  £  Pennings,  1976;  Steers,  Porter, 
Mowday,  £  Stone,  1975;  Price,  1968;  Coulter,  1979;  Mahoney  £  Frost,  1974) 
The  dimensions  of  effectiveness  growing  out  of  several  dozen  models  were 
cataloged  by  Campbell  et  al.,  1974.  The  dimensions  are: 

Overall  Effectiveness  Conflict /Cohesion 


Overall  Effectiveness 

Productivity 

Efficiency 

Profit 

Quality 

Accidents 

Growth 


Absenteeism 

Turnover 

Satisfaction 

Motivation 

Morale 


Control 


Flexibility/Adaptation 

Goal  Consensus 

Role  and  Norm  Congruence 

Managerial  Task  Skills 

Managerial  Interpersonal  Skills 

Information  Management  and 
Communications 

Readiness 

Utilization  of  Environment 
Evaluation  of  External  Entities 
Stability 

Internalization  of  Organiza¬ 
tional  Goals 

Value  of  Human  Resources 


In  thinking  about  organizational  effectiveness,  individuals  (e.g., 
managers,  researchers)  usually  have  their  own  individual  models  which  may 
include  one  or  more  of  the  listed  dimensions.  Rather  than  attempt  to  de¬ 
fine  effectiveness,  Campbell  et  al.  state: 

Perhaps  a  better  way  to  think  of  organizational 
effectiveness  is  as  an  underlying  construct  which 
has  no  direct  operational  definition,  but  which 
constitutes  a  model  or  theory  of  what  organiza¬ 
tional  effectiveness  is  (1974,  p.5). 

Following  this  approach,  Campbell,  et  al .  (1974)  describe  two  basic  types 
of  models  of  effectiveness  which  they  label  goal  centered  and  natural 
systems.  In  a  separate  paper.  Coulter  (1979)  defines  three  types  of  ef¬ 
fectiveness  models:  behavioral-attitudinal ,  process  and  goal  attainment. 
Coulter  illustrates  the  models  with  examples  of  the  types  of  dimensions 
used  to  assess  effectiveness.  Advocates  of  the  behavioral-attitudinal 
model  claim  that  certain  behavioral  and  attitudinal  characteristics  of 
individuals  and  groups  of  individuals  offer  the  most  percise  measures  of 
the  effectiveness  of  organizations.  Included  among  these  are  absence  of 
tension  and  conflict,  employee  satisfaction,  psychological  commitment, 
turnover  and  absenteeism,  inter-personal  relations,  and  morale.  The  pro¬ 
cess  models  involve  one  or  more  variables  which  deal  with  the  internal 
operation  of  the  organization  (e.g.,  flexibility,  adaptability,  creativity, 
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open  communications,  personnel  acquisition,  retention,  and  utilization) 
or  the  linkage  between  the  organization  and  its  environment  (e.g.,  con¬ 
trol  over  the  environment,  ability  to  acquire  valued  and  scarce  re¬ 
sources)  .  The  goal  attainment  model  defines  effectiveness  as  the  extent 
to  which  an  organization  achieves  its  goals.  While  this  definition  seems 
straightforward  enough,  there  is  usually  considerable  disagreement  among 
researchers  over  how  to  define  goals  and  how  to  measure  goal  achievement. 
This  argument  follows  the  lines  of  whether  the  goals  are  those  prescribed 
by  decision  makers,  those  derived  from  some  theory  of  organizations,  or 
the  informal  goals  which  "actually”  guide  the  behavior  of  organizational 
members.  Questions  also  arise  as  to  whether  goal  attainment  is  to  be 
measured  through  employee  ratings,  supervisory  ratings,  customer  ratings, 
or  through  the  use  of  archival  data.  This  discussion  recognizes  that  ef¬ 
fectiveness  can  be  viewed  from  the  perspective  of  at  least  three  differ¬ 
ent  groups,  owner/manager,  employees,  and  the  society  at  large. 

Mahoney  and  Frost  (1974)  raise  the  issue  that  the  choice  of  a  model 
of  effectiveness  to  apply  to  a  given  organization  may  be  a  function  of 
the  nature  of  its  technology.  These  authors  studied  386  organizational 
units.  With  respect  to  the  dominant  technology  employed  by  the  organiza¬ 
tional  unit,  units  were  classified  as  either  "long-linked"  (59%) ,  "medi¬ 
ating"  (24%),  or  "intensive"  (17%).  These  were  defined  as  follows: 

1.  long-linked  —  Organizations  in  which  there  is  a  belief 

of  complete  cause/effect  knowledge,  high 
predictability  of  both  input  and  outputs, 
and  crystallized  norms. 

2.  mediating  —  When  cause/effect  knowledge  and  predicta¬ 

bility  of  outputs  are  conditional,  where 
some  discretion  must  be  used  to  match  in¬ 
put  with  available  programs  for  processing. 

3.  intensive  --  Belief  in  incomplete  cause/effect  knowl¬ 

edge,  ambiguous  standards  of  desirability, 
low  predictaDility  of  outcome  of  actions 
and  a  high  deqree  of  discretion  required. 

Managers  one  organizational  level  above  these  units  were  asked  to  rate 
each  of  the  organizational  units  in  terms  of  the  following  14  dimensions 
of  organizational  effectiveness  as  well  as  overall  effectiveness. 

1.  Performance.  Efficient  performance,  mutual  support  by 

supervisors  and  subordinates,  utilization  of  personnel 

skills  and  abilities. 

2.  Planning.  Avoiding  disruption  and  lost  time  through 

scheduling  and  coordination. 

3.  Supervisory  control.  Extent  of  supervisory  control 

over  work  progress. 

4.  Results  emphasis.  The  organization  emphasizes  results 

not  procedures. 

5.  Cooperation.  The  extent  to  which  scheduled  commitments 

to  other  units  are  met. 


6.  Flexibility.  The  ability  of  the  group  to  adapt  to 

change  and  new  ideas. 

7.  Initiation.  Takes  action  to  initiate  improvements 

in  work  methods  and  operations,  makes  suggestions. 

8.  Coordination.  Coordinates  and  relates  activities 

with  other  units  in  advance. 

9.  Conflict.  Avoidance  of  conflict  with  other  units 

regarding  responsibilities  and  authority. 

10.  Reliability.  Meets  objectives  without  constant 

need  for  monitoring  and  followup. 

11.  Development.  The  extent  of  involvement  in  training 

and  development  activities. 

12.  Staffing.  Ability  and  willingness  of  staff  to  move 

among  assignments,  and  the  extent  of  promotion 

from  within. 

13.  Delegation.  Degree  to  which  supervisors  effectively 

delegate  work  to  subordinates. 

14.  Cohesion.  Lack  of  complaints,  grievances,  and 

conflict  within  unit. 

15.  Overall  organizational  effectiveness.  (Adapted  from 

Mahoney  and  Frost,  1974,  p.  129) . 

Regression  analyses  used  to  capture  the  policies  of  managers  revealed 
differences  in  the  "models  of  effectiveness"  held  by  managers  as  a  func¬ 
tion  of  the  type  of  technology  employed  by  the  organizational  unit.  These 
differences  were  not  always  consistent.  In  general,  however,  for  long- 
linked  technologies,  the  criteria  of  effectiveness  were  smoothness  of 
operation  (planning) ,  output  performance  (results  emphasis) ,  and  reliabil¬ 
ity  of  performance  (reliability) .  For  mediating  technologies,  the  pre¬ 
dominant  criteria  appear  to  be  adaptation  to  change  (flexibility) ,  smooth¬ 
ness  of  operation  (planning) ,  output  performance  (results  emphasis) ,  and 
supervisory  control.  In  the  case  of  intensive  technologies  still  other 
dimensions  emerged.  The  predominant  criteria  of  intensive  technologies 
were  performance,  meeting  work  commitments  to  other  units  (cooperation) , 
and  quality  of  staff  (development  and  staffing) . 

This  research  is  quite  significant  in  pointing  out  that  definitions 
of  organizational  effectiveness  and  the  criteria  for  measuring  organiza¬ 
tional  effectiveness  appear  to  be  conditional.  At  least  from  the  point 
of  view  of  managers  who  must  judge  the  effectiveness  of  organizational 
units,  the  criteria  on  which  these  judgements  are  made  vary  depending  on 
the  type  of  work  performed  by  the  organization. 

Effectiveness  and  Productivity 

The  distinction  between  effectiveness  and  productivity  remains  con¬ 
fused.  In  attempting  to  clarify  this  relationship  there  appear  to  be 
three  viewpoints  expressed  in  the  literature:  (a)  productivity  is  a  dimen- 
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sion  of  the  broader  concept  of  effectiveness,  (b)  productivity  is  a  broad 
concept  which  encompasses  both  efficiency  and  effectiveness,  (c)  produc¬ 
tivity  and  effectiveness  are  separate  but  related  concepts. 

In  their  review,  Campbell  et  al.  (1974)  distinguished  between  pro¬ 
ductivity  and  efficiency  and  considered  both  to  be  dimensions  of  organi¬ 
zational  effectiveness.  Productivity  was  defined  in  terms  of  "the  quan¬ 
tity  or  volume  of  the  major  product  or  service  that  the  organization  pro¬ 
vides..."  (Campbell  et  al.,  1974,  p.  50).  This  definition  of  productiv¬ 
ity  might  be  more  accurately  termed  "production"  since  it  takes  no  account 
of  the  resources  required  to  produce  the  product  or  service.  This  is  not 
to  fault  these  authors  since  they  are  simply  reflecting  the  confusion 
between  productivity  and  production  that  is  rampant  in  the  literature. 
"Efficiency  is  usually  thought  of  in  terms  of  a  ratio  that  reflects  a 
comparison  of  some  aspect  of  unit  performance  to  the  costs  incurred  for 
that  performance"  (Campbell  et  al.,  1974,  p.  56).  As  defined,  efficiency 
seems  synonymous  with  the  economist's  definition  of  productivity. 

A  second  view  of  the  relationship  between  productivity  and  effective¬ 
ness  is  expressed  by  Coulter  (1979) .  Most  simply  stated,  this  view  is 
that  "productivity  takes  into  account  the  efficiency  with  which  an  organ¬ 
ization  achieves  its  level  of  effectiveness"  (Coulter,  1979,  p.  80) .  In 
this  view,  productivity  is  the  integrating  concept  that  links  effective¬ 
ness  and  efficiency.  A  similar  position  has  been  taken  by  Balk  (1975), 
who  discusses  productivity  as  a  systems  concept.  He  distinguishes  between 
efficiency  (the  ratio  of  output  to  input)  and  effectiveness  (the  ratio  of 
output  to  a  standard;  e.g.,  quality,  timeliness).  Balk  then  defines  pro¬ 
ductivity  as  follows  (1975,  p.  130)-. 

Productivity  =  efficiency  and  effectiveness 

or 

Productivity  =  0/1  +  O/S  where  O  =  output 

I  =  input 

S  =  "Standard"  for 

measuring  output. 

A  similar  view  of  productivity  as  a  composite  of  efficiency  and  effective¬ 
ness  is  presented  in  a  handbook  on  Total  Performance  Management  (TPM) 
(National  Center  for  Productivity  and  Quality  of  Working  Life,  1978, 
p.  ID  : 

Productivity  in  the  private  sector  is  usually  defined 
as  a  ratio — output  per  unit  of  input.  This  definition 
is  deceptively  simple,  however.  Disagreements  still 
occur  over  many  basic  questions:  Is  it  really  possible 
to  measure  productivity?  How  do  you  take  into  account 
the  variety  of  tasks  within  one  job?  How  can  you  separate 
the  contributions  of  staff  hours  from  other  inputs,  such 
as  capital  investment? 

In  the  public  sector,  the  definition  of  productivity  en¬ 
compasses  two  components,  efficiency  and  effectiveness. 
Efficiency  measurements  assess  the  cost  per  unit  of  output. 
Usually  quantitative  in  nature,  efficiency  measures  typically 
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use  staff  hours  as  the  basis  for  input  measures  and 
quantity  produced  as  output  measures.  A  more  sophis¬ 
ticated,  but  more  difficult,  input  measurement  is 
total  resource  expenditures,  which  may  include  energy 
use,  capital,  investment,  depreciation,  overhead,  and 
other  contributing  inputs. 

As  this  discussion  points  out,  productivity,  viewed  as  output/input, 
is  the  predominant  view  in  the  private  sector.  Such  a  notion  grew  out  of 
manufacturing  operations  employing  "long-linked"  technologies.  However, 
in  the  public  sector  (as  well  as  in  private  sector  service  and  administra¬ 
tion  units)  this  model  is  not  adequate  to  account  for  productivity. 

Models  of  productivity  incorporating  both  effectiveness  as  well  as  effi¬ 
ciency  are  needed  in  the  public  sector  and  also  in  non-manufacturing 
private  sector  organizations. 

A  third  view  of  the  relationship  between  effectiveness  and  productiv¬ 
ity  has  been  proposed  by  Price  (1977).  In  his  discussion.  Price  equates 
productivity  and  efficiency.  He  says  that  productivity  (efficiency)  is 
separate  from,  but  related  to  effectiveness.  Effectiveness  is  defined  by 
Price  as  the  degree  to  which  an  organization  achieves  its  goals  (Price, 
1977) .  To  illustrate  the  meaning  of  effectiveness,  Price  cites  several 
examples.  A  mental  hospital  with  a  therapeutic  goal  that  successfully 
releases  a  high  proportion  of  its  patients  into  the  community  would  be  an 
effective  hospital.  However,  as  Price  points  out,  effectiveness  and  pro¬ 
ductivity  (efficiency)  can  vary  independently.  For  example,  a  business 
firm  whose  goal  is  profits  may  be  very  productive  but,  due  to  a  declining 
market  for  its  output,  suffer  from  low  profits.  Despite  this  firm's  high 
level  of  productivity  (efficiency) ,  it  is  not  effective  (Price,  1977, 

pp.  110-111) . 

Price  (1977)  takes  a  narrow  view  of  productivity  compared  to  Coul¬ 
ter's  (1979)  broader  view.  Price  would  not  accept  Coulter's  notion  that 
productivity  encompasses  both  efficiency  and  effectiveness.  This  is  the 
crux  of  the  definitional  problem.  There  are  two  questions  that  must  be 
answered  to  resolve  this  issue.  First,  should  productivity  be  synonymous 
with  efficiency  or  is  it  a  broader  concept  which  includes  both  efficiency 
and  effectiveness?  Secondly,  if  it  is  narrowly  defined,  is  productivity 
a  dimension  of  effectiveness,  or  are  productivity  and  effectiveness  to¬ 
tally  separate,  but  related  concepts? 

Quality  of  Working  Life 

The  preceding  discussion  points  out  the  difficulties  in  distinguish¬ 
ing  between  the  concepts  of  effectiveness  and  productivity.  A  related 
problem  exists  with  another  "concept"  freqently  appearing  in  current  or¬ 
ganizational  literature — quality  of  working  life.  Quality  of  working  life 
is  a  concept  which  has  almost  as  many  definitions  as  organizational  effec¬ 
tiveness.  For  present  purposes,  "quality  of  work,"  "quality  of  work  life," 
and  "quality  of  working  life"  are  considered  synonymous. 

A  leading  proponent  of  quality  of  working  life,  Ted  Mills,  defines 
the  concept  as  an  attempt: 
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to  provide  people  at  work  (manager,  supervisors,  rank 
and  file  workers)  with  structured  opportunities  to  be¬ 
come  actively  involved  in  a  new  interpersonal  process 
of  problem  solving  toward  both  a  better  way  of  working 
and  a  more  effective  work  organization,  the  payoff  from 
which  includes  the  best  interests  of  employees  and  em¬ 
ployers  in  equal  measure  (Mills,  1978,  p.  23) . 

In  their  extensive  literature  review,  Katzell  et  al.  (1975) 
define  quality  of  working  life  as  the  combination  of  a  worker’s  job  satis¬ 
faction,  motivation,  and  balance  of  work  life  with  other  aspects  of  the 
worker's  life. 

A  worker  can  be  said  to  enjoy  a  high  quality  of  working 
life  when  he  (a)  has  positive  feelings  towards  his  job 
and  its  future  prospects,  (b)  is  motivated  to  stay  on 
the  job  and  perform  well,  and  (c)  feels  his  working  life 
fits  well  with  his  private  life  to  afford  him  a  balance 
between  the  two  in  terms  of  his  personal  values  (Katzell 
et  al.  1975,  pp.  69-70)  . 

Katzell  et  al .  (1975)  view  quality  of  working  life  somewhat  more 
broadly  than  Mills  (1978)  .  Furthermore,  the  Mills  definition  views  r  ial- 
ity  of  working  life  as  a  process  while  Katzell  et  al.  (1975)  view  it 
as  the  individual's  evaluation  of  the  outcomes  of  the  work  relationship. 

A  third  perspective,  and  one  which  lends  itself  to  measurement,  is 
proposed  by  Herrick  and  MacCoby  (1972) .  This  approach  defines  "quality 
of  work"  as  comprised  of  four  dimensions:  security,  equity,  individua¬ 
tion,  and  democracy.  These  are  defined  as  follows: 

1.  Security  —  Freedom  from  fear  and  anxiety  concerning 
physical  health  and  safety,  income,  and  future  employment. 

2.  Equity  —  The  extent  to  which  compensation  received 
is  commensurate  to  the  worker's  contribution  to  the 
value  of  a  service  o’-  product. 

3.  Individuation  (craftsmanship,  autonomy  and  learning)  — 

The  extent  to  which  work  stimulates  the  development  of 
unique  abilities,  and  capacity  for  craftsmanship,  and 
continued  learning  rather  than  boredom  and  stagnation. 

4.  Democracy  —  The  extent  to  which  workers'  views  are 
listened  to  and  taken  into  account  in  decision  making, 
and/or  structures  are  created  in  which  workers'  power 
and  responsibility  are  institutionalized.  "Wherever 
technically  possible,  workers  should  manage  themselves..." 
(Herrick  &  MacCoby,  1972.  In  Herrick,  1975,  p.  4) . 

Herrick  (1975)  assumes  that  organizations  can  be  assessed  and  pro¬ 
filed  on  each  of  these  four  dimensions.  In  this  framework,  changes  in 
"quality  of  work"  are  hypothesized  to  relate  to  changes  in  organizational 
outcomes  including  labor  productivity,  absenteeism,  turnover,  scrap,  etc. 
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A  fourth  view  of  the  concept  of  quality  of  working  life  is  provided 
by  the  report  of  a  task  force  established  to  develop  methods  for  measuring 
quality  of  work  life  (Davis  &  Cherns,  1975) .  The  goal  of  the  task  force 
was  to  review  the  various  approaches  to  defining  and  measuring  quality  of 
working  life  and  to  recommend  a  definition  and  a  set  of  measures  that 
would  have  utility  for  assessment  and  policy  making.  After  acknowledging 
lack  of  agreement  among  researchers  and  practitioners,  the  group  proposed 
a  number  of  dimensions  which  should  be  included  in  any  conceptualization 
of  quality  of  working  life  (Davis  &  Cherns,  1975) : 

1.  Employment  conditions  (safety,  health,  physical  environment) 

2.  Employment  security  (  future  of  the  job) 

3.  Income  adequacy  (present  and  future) 

4.  Equity  of  pay  and  other  rewards 

5.  Worker  autonomy 

6.  Social  interaction 

7.  Self  Esteem 

8.  Democracy  (participation  in  decision  making) 

9.  Worker  satisfaction 

These  four  views  of  quality  of  working  life  share  the  basic  ingredi¬ 
ent  of  workers  having  input  into  decisions  that  affect  them  at  work.  The 
Mills  (1978)  definition  emphasizes  process  issues  while  the  others,  Kat- 
zell  et  al .  (1975),  Herrick,  (1975)  and  Davis  and  Cherns  (1975)  emphasize 
the  individual's  subjective  reaction  to  conditions  in  the  work  place. 
Katzell  et  al.  (1975)  offer  a  broad  definition  that  allows  for  variation 
in  employee  values  while  Herrick  favors  a  definition  which  is  normative 
and  prescriptive  with  respect  to  "good"  quality  of  working  life.  Davis 
and  Cherns  add  a  future  orientation  to  the  definition.  They  view  quality 
of  working  life  not  only  to  include  worker  reactions  to  present  conditions 
but  also  their  reaction  to  anticipated  future  developments. 

Quality  of  Working  Life  and  Productivity 

Either  implicitly  or  explicitly,  all  of  these  approaches  view  quality 
of  working  life  as  conceptually  separate  from  objective  outcomes  of  the 
workplace  such  as  productivity.  However,  all  would  admit  that  there  is 
some  relationship  between  "good"  quality  of  working  life  and  labor  pro¬ 
ductivity.  For  instance,  Herrick  (1975)  states  this  in  clear  terms: 

The  concept,  however,  rests  primarily  on  the  belief  that 
the  actions  which  can  reasonably  be  expected  to  insure 
labor  productivity  are  the  same  actions  which  should  be 
taken  in  any  event  to  increase  human  fulfillment  through 
work:  that  is,  improving  the  conditions  of  security  and 

equity  and  the  opportunities  for  individuation  and  parti¬ 
cipation  (p.  18) . 

The  conclusion  emerging  from  this  review  is  that  quality  of  working 
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life  and  productivity  are  conceptually  distinct,  but  compatible  concepts. 
Most  authors  would  agree  that  productivity  improvements  frequently  accom¬ 
pany  improvements  in  quality  of  working  life.  Furthermore,  as  stated  by 
Herrick  (1975) ,  procedures  taken  by  organizations  to  improve  quality  of 
working  life  may  be  very  similar,  or  identical,  to  the  procedures  required 
to  improve  labor  productivity.  The  philosophical  difference  appears  to  be 
that  quality  of  working  life  advocates  have  concentrated  more  on  outcomes 
to  employees  while  productivity  advocates  have  focused  more  on  benefits 
to  the  organizations. 


Summary 

This  chapter  has  surveyed  various  definitions  of  productivity.  Def¬ 
initions  were  classified,  for  discussion  purposes,  as  representative  of 
several  viewpoints:  economists,  engineers,  accountants,  managers,  and 
industrial/organizational  psychologists.  The  chapter  concluded  with  a 
discussion  of  two  related  concepts,  organizational  effectiveness  and 
quality  of  working  life.  These  were  described  and  their  assumed  re¬ 
lationships  to  productivity  were  presented. 
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ITT.  PRODUCTIVITY  -  AN  AIR  FORCE  VIEW 


This  chapter  will  describe  the  "official"  Air  Force  view  of  produc¬ 
tivity  proposed  by  Air  Force  Headquarters  and  selected  Major  Commands. 

Next,  it  will  present  some  "unofficial"  views  derived  from  interviews  with 
Air  Force  managers  and  researchers  in  a  wide  variety  of  field  organiza¬ 
tions.  The  interviews  were  conducted  during  the  period  April  1979  -  June 
1980,  and  involved  over  50  organizations  and  over  100  interviewees.  Fi¬ 
nally,  this  chapter  will  present  an  approach  to  conceptualizing  produc¬ 
tivity  in  Air  Force  organizations  that  recognizes  the  unique  mission  of  a 
military  organization. 

Productivity  -  The  Official  Air  Force  View 

On  15  November  1979,  the  Air  Force  issued  its  Productivity  Plan. 

This  plan  was  issued  in  compliance  with  DoD  Instruction  5010.34  (August  4, 
1975)  and  DoD  Directive  5010.31  (April  27,  1979)  which  directs  each  DoD 
Component  to  establish  its  own  productivity  improvement  goals  and  planned 
approaches  to  productivity  enhancement.  In  these  documents  the  DoD  pro¬ 
vides  three  different  definitions  of  productivity: 

1.  Organizations  must  be  (a)  ef fective--accomplish  the  right 
things,  in  the  right  quantities,  at  the  right  times,  and 
(b)  efficient — accomplish  the  right  things  with  the  lowest 
possible  expenditure  of  resources.  The  efficiency  with 
which  organizations  utilize  all  types  of  fund  resources 
(operating  and  investment)  to  accomplish  their  mission 
represents  total  resource  productivity.  The  efficiency 
with  which  organizations  utilize  labor  resources  to 
accomplish  their  missions  reppigsents  labor  productivity 
(DoD  1-5010.34,  August  4,  1975). 

2.  The  productivity  of  an  organization  may  be  broadly  defined 
as  the  efficiency  with  which  its  resources  are  utilized  to 
produce  final  outputs  (DoD  1-5010.34,  August  4,  1975,  Enel. 

3  p. a . ) . 

3.  Productivity.  The  ratio  of  goods  produced  or  services 
rendered  (output)  to  resources  expended  (input)  (DoD 
D-5010.31,  April  27,  1979). 

The  first  example  defines  productivity  in  terms  of  both  efficiency  and 
effectiveness.  However,  the  second  and  third  define  productivity  as 
roughly  comparable  to  efficiency.  Thus,  the  DoD  definition  of  productiv¬ 
ity  is  unclear. 

In  its  plan,  the  Air  Force  adopted  the  narrow,  efficiency  definition. 
Productivity  is  defined  as  "the  ratio  of  goods  produced  or  services  ren¬ 
dered  (output)  to  resources  expended  (input)"  (Air  Force  Productivity  Plan, 
15  November  1979) .  While  the  plan  acknowledges  the  concept  of  effective¬ 
ness  and  effectiveness  measurement  as  a  way  to  determine  if  a  particular 
goal  or  objective  is  achieved,  the  focus  of  the  plan  is  clearly  on  ef¬ 
ficiency. 
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The  efficiency  view  of  productivity  is  also  reflected  in  the  defini¬ 
tion  of  productivity  contained  in  AFR  25-5,  the  regulation  describing  the 
Air  Force  Management  Engineering  Program.  In  the  regulation's  glossary 
of  standard  terms,  productivity  is  defined  as: 

The  efficiency  in  using  resources  to  carry  out  a 
given  mission.  Measured  in  terms  of  the  ratio  of 
output  to  input.  Also,  the  ratio  of  actual  per¬ 
formance  to  standard  performance,  or  level  of 
performance,  in  a  previous  period — applicable  to 
either  an  individual  worker  or  a  group  of  workers 
(AFR  25-5,  Vol.  1,  7  November  1977,  p.  A2-3) . 

While  this  definition  addresses  mission  accomplishment,  an  effectiveness 
concept,  it  emphasizes  efficiency  as  the  way  to  operationalize  the  con¬ 
cept,  e.g.,  output/input ,  or  actual  output  compared  to  a  standard. 

Another  "official"  view  of  productivity  comes  from  AFM  25-1,  The  USAF 
Management  Process,  15  October  1964. 

The  effectiveness  of  the  Air  Force  is  its  capability, 
on  an  ever-ready  basis,  to  meet  the  threat  of  any 
combination  of  potential  enemies.  This  capability 
is  the  end-result  of  the  overall  operational  system 
of  the  Air  Force;  the  numerous  components  of  the 
system  are  the  achievements  of  specific  objectives. 

The  need  to  maintain  our  capabilities  on  a  con¬ 
tinuing  basis  is  nearly  always  a  paramount  factor  in 
our  managerial  efforts  to  increase  the  quantity  and 
quality  of  the  end-product.  Except  in  emergencies, 
managers  should  increase  effectiveness  only  to  the 
level  at  which  it  can  be  maintained.  Managerial 
action  which  increases  productivity  but  hurts  morale, 
for  example,  is  likely  to  be  so  destructive  to  long¬ 
term  effectiveness  that  it  cancels  any  short-term  gain. 

An  improvement  which  undermines  our  capabilities  to 
sustain  effectiveness  is  no  improvement  at  all  (p.  7). 

The  precise  meaning  of  productivity  in  the  quoted  passage  is  not 
clear.  What  is  interesting,  however,  is  the  assertion  that  "productivity" 
and  effectiveness  levels  need  to  be  kept  in  balance.  Attempts  to  maximize 
productivity  in  the  short-term  may  lead  to  a  long-run  decrease  in  organi¬ 
zational  effectiveness.  Throughout  AFM  25-5  there  are  references  caution¬ 
ing  against  the  tendency  toward  "piece-meal  management"  or  attempts  to 
substitute  "technical  efficiency"  for  overall  operational  effectiveness 
(pp.  36-37,  p.  40) . 

As  Major  Commands  (MAJCOMs)  and  Operating  Agencies  are  asked  to  com¬ 
ply  with  the  provisions  of  the  Air  Force  Productivity  Plan  they  must  ad¬ 
dress  the  question  of  how  to  define  productivity.  As  previously  dis¬ 
cussed,  there  is  some  room  for  interpretation  with  respect  to  the  official 
Air  Force  definition  of  productivity,  although  the  efficiency  definition 
appears  to  be  the  predominant  one.  Some  of  the  definitions  proposed  by 
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the  MAJCOMs  in  their  own  plans  include: ^ 

Productivity  is  the  we 11 -documented,  more 
efficient  use  of  all  resources  to  accomplish 
a  stated  mission. 

Productivity.  The  relationship  between  the 
volume  of  goods  and  services  produced  and  the 
physical  input  used  in  their  production  over 
a  specified  period  of  time. 

Productivity.  The  efficiency  with  which  re¬ 
sources  are  used  to  accomplish  a  given  mission. 

. . .doing  more  with  less  and  insuring  mission 
accomplishment  with  minimum  resources. 

Based  on  this  sample  of  definitions,  there  appears  to  be  a  tendency 
for  the  MAJCOMs  to  include  explicit  reference  to  "mission  accomplishment" 
rather  than  "goods  and  services  produced"  in  their  definitions  of  produc¬ 
tivity.  However,  in  operationalizing  productivity  and  reporting  to  higher 
headquarters,  MAJCOMs  are  under  pressure  to  measure  and  report  efficiency 
defined  as  output  of  final  goods  and  services  per  unit  of  labor  input. 
There  is  no  corresponding  requirement  in  the  Air  Force  Plan  to  measure  or 
report  the  impact  of  this  production  of  goods  and  services  on  mission 
accompl ishment .  While  unlikely,  it  is  conceivable  that  the  "more  produc¬ 
tive"  organization  could  be  the  least  effective  in  terms  of  overall  mis¬ 
sion  accomplishment  and  vice  versa.  Because  MAJCOMs  seem  principally 
concerned  with  mission  accomplishment,  the  pressure  to  measure  and  report 
efficiency  separate  from  mission  accomplishment  places  them  in  a  difficult 
position.  In  order  to  provide  a  better  understanding  of  this  situation, 
the  next  section  provides  some  reactions  to  Air  Force  productivity  initia¬ 
tives  obtained  from  Air  Force  managers  during  the  series  of  field  visits. 

Air  Force  Productivity  -  A  View  From  the  Field 

In  interpreting  the  reactions  of  Air  Force  managers  to  the  concept  of 
productivity,  one  should  kf ep  in  mind  that  during  recent  years  the  Air 
Force  has  been  shrinking  in  terms  of  active  duty  personnel.  "Doing  more 
with  less"  has  become  the  slogan  associated  with  these  cutbacks  and  also 
the  target  of  many  managers'  wrath.  Unfortunately,  the  productivity  pro¬ 
gram,  perhaps  partly  due  to  its  organizational  identity  with  the  manpower 
community  which  is  fierce ived  as  responsible  for  manpower  cuts,  has  become 
identified  with  budget  and  management  cuts. 

During  the  course  of  this  proiect,  field  visits  were  made  to  MAJCOMs 
and  operating  agencies.  More  than  50  organizations  were  visited  and  in¬ 
terviews  were  conducted  with  over  twice  that  many  people.  One  of  the 
questions  asked  was,  "What  does  the  term  productivity  mean  to  you?"  A 
variety  of  responses  were  obtained. 


1These  definitions  were  included  in  drafts  of  Command  Plans  provided  the 
author  on  the  condition  that  their  exact  source  not  be  disclosed. 
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From  the  Air  Force  Logistics  Command  (AFLC),  a  representative  sampling 
of  the  definitions  obtained  is  as  shown: 

Productivity  is  the  ratio  of  output  to  input. 

Input  should  include  labor  and  other  quanti¬ 
fiable  variables  (e.g.,  energy,  raw  materials, 
equipment,  etc.).  Quantifying  output  is  a 
horrendous  problem.  Operationally  defining 
effectiveness  is  a  challenge  that  is  unmet. 

Productivity  is  the  most  effective  and 
efficient  use  of  time  and  resources. 

Productivity  in  distribution  (supply)  can  be 
defined  as  the  number  of  line  items  issued 
or  received  per  man  hour  worked. 

These  definitions  reflect  the  relatively  concrete,  measurable  types 
of  work  performed  in  AFLC .  While  there  are  still  difficulties  associated 
with  output  measurement,  this  command  comes  closest  to  being  an  industrial 
operation  employing  "manufacturing  like"  technology.  In  addition,  compared 
to  other  MAJCOMS,  AFLC  has  had  the  most  experience  in  applying  productivity 
concepts  to  its  operations. 

When  other  MAJCOMs  are  considered,  a  somewhat  different  situation  is 
discovered.  Consider  the  following  definition  of  productivity  from  a  man¬ 
ager  in  Air  Training  Command  (ATC) . 

The  big  area  in  ATC  is  cost  per  graduate.  However, 
we  don't  know  if  this  is  productive.  It  fails  to 
address  quality.  The  real  measure  of  productivity 
in  training  is  not  whether  a  person  graduates  or 
how  much  it  costs,  but  what  that  person  does  3-5 
years  from  now  after  he  gets  out  into  the  field. 

Another  ATC  manager  describe?  his  frustrations  with  productivity  measure¬ 
ment  as  now  practiced  in  the  Air  Force: 

...if  we're  going  to  talk  about  productivity  and  how 
we  go  about  it,  we  need  to  have  some  kind  of  conceptual 
model  of  how  we  approach  it.  At  the  top  of  the  list  is 
an  interpretation  of  the  mission  in  terms  of  objectives. 

At  the  tail  end,  we  need  to  measure  how  well  these  ob¬ 
jectives  are  acheived  and  what  kinds  of  measures  are 
necessary  to  achieve  those  objectives.  ...Since  we  don't 
have  to  make  some  sort  of  "sales  bogey,"  it's  difficult 
to  translate  the  total  mission  statement  into  specific 
objectives  that  not  only  can  be  measured  but  are  worth 
measuring.  We  measure  the  things  we  can  measure  but  the 
greatest  part  of  a  mission  is  constituted  by  things  that 
are  not  easy  to  measure ... [we]  measure  those  things  that 
are  easy  to  measure — and  we  measure  them  just  great--but 
whether  we  achieve  our  mission  will  depend  on  these  other 
objectives  that  we've  avoided  because  we're  not  sure  how 
to  measure  them.  Until  we  come  to  grips  with  this  sort 
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of  thing,  we're  not  sure  we're  assessing  the  right  thing. 

And  if  you  don't  assess  the  right  thing  you'll  never  know 
if  you've  been  productive. 

Concern  with  mission  was  also  reflected  in  the  productivity  defini¬ 
tions  offered  by  managers  from  Strategic  Air  Command  (SAC) . 

It  seems  that  organizations  have  to  determine  effective¬ 
ness  first  then  define  objectives  and  then  determine  how 
efficiently  these  objectives  are  being  carried  out. 

Maybe  one  of  our  problems  is  that  our  mission  is  not 
defined  specifically  enough.  I've  looked  at  a  lot  of 
mission  statements  down  to  the  squadron  level  and 
they're  all  very  vague  and  general  in  nature. 

Another  SAC  commander  addressed  the  issue  of  the  level  of  the  organization 
as  a  variable  affecting  the  definition  of  productivity. 

In  order  to  define  productivity  one  must  first  specify 
the  level  we  are  defining  productivity  for.  In  other 
words,  the  measure  of  productivity  for  a  squadron  is 
different  from  that  for  a  numbered  Air  Force. 

The  dilemma  facing  one  who  attempts  to  define  productivity  for  an 
operational  organization  is  reflected  in  this  statement  from  a  SAC 
manager. 

For  input  we  have  117,000  people  in  SAC,  we  put  billions 
of  dollars  into  deterrence  (equipment) ,  we  have  buildings 
and  facilities  at  76  bases  spread  all  over  the  world.  But 
what  is  our  output?  The  fact  that  we  didn't  go  to  war  last 
night?  The  fact  that  our  missiles  are  all  on  alert  and  all 
the  bombers  are  ready  to  go?  What  is  output  for  SAC? 

Air  Force  Productivity  -  A  Researcher's  View 

Air  Force  researchers  and  those  doing  research  in  other  military  ser¬ 
vices  shed  a  slightly  different  perspective  on  the  problem  of  defining 
productivity.  Reflecting  more  of  a  theoretical  orientation  than  most 
managers,  but  generally  a  more  pragmatic  view  than  their  civilian  academic 
counterparts,  military  researchers  provide  an  important  source  of  ideas 
for  any  attempt  to  define,  measure  and  improve  Air  Force  productivity. 
Researchers  reflect  the  disciplines  they  represent  and,  therefore,  do  not 
speak  with  one  voice.  This  section  will  consider  a  range  of  views  on  the 
meaning  of  productivity  suggested  by  members  of  the  military  research 
community. 

In  a  recent  paper  which  reviewed  the  meanings  of  productivity  and 
effectiveness,  Engel  (1979)  draws  the  following  conclusion. 

Therefore,  productivity  is  one  of  the  objectives  of  an 
organization  but  must  be  compromised  for  the  overall 
optimal  organizational  effectiveness.  The  other  ob¬ 
jectives  with  which  productivity  competes  differ  from 
organization  to  organization  along  with  the  relative 
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weights  assigned  to  these  individual  objectives.  There 
have  been  many  models  for  measuring  organizational  effec¬ 
tiveness  but  there  has  bec  ■  little  consistency  in  the 
objectives  considered  important  to  overall  effectiveness. 

In  the  military  establishment’s  continuing  struggle  to  do 
more  with  less,  extreme  care  must  be  token  not  to  develop 
measures  of  "success"  and  then  to  apply  them  to  all  organ¬ 
izations.  Since  maximization  of  all  goals  is  seldom  if 
ever  possible,  a  trade-off  proposition  is  frequently  re¬ 
quired.  When  the  attainment  of  one  objective  increases, 
there  is  usually  a  decrease  in  some  other  area.  Better 
management  will  result  from  a  better  understanding  of 
these  concepts  (p.  19) . 

Engel  's  conclusion  is  similar  to  that  drawn  in  the  excerpt  from  AFM  25-1 
quoted  previously,  namely  that  attempts  to  maximize  productivity  (effi¬ 
ciency)  will  likely  damage  overall  organizational  effectiveness.  There¬ 
fore,  organizations  should  seek  to  "optimize”  both  productivity  and  ef¬ 
fectiveness.  In  the  long  run  this  is  the  ^nly  way  to  sustain  either 
efficiency  or  effectiveness. 

In  a  research  report  investigating  the  impact  of  the  Program  Planning 
and  Budgeting  (PPB)  System  on  the  Air  Force  operating  manager,  Letzkus 
(1973)  observed: 

A  comparison  of  actual  and  planned  results  is  undimensional 
in  that  it  addresses  only  the  effectiveness  of  mission  op¬ 
erations.  Missing  from  such  an  evaluation  is  the  test  of 
operating  efficiency.  Although  actual  operating  results 
may  exceed  planned  results,  at  what  cost  was  this  increased 
effectiveness  achieved?  Conversely,  the  fact  that  actual 
operating  results  are  less  than  planned  results  may  be 
counterbalanced  by  increased  operating  efficiency.  Even 
granting  the  preeminence  of  mission  effectiveness,  the 
fact  remains  that  performance  measurement  must  be  two 
dimensional:  i.e.,  it  must  encompass  the  criteria  of 

both  mission  effectiveness  and  mission  efficiency  (pp.  187- 
189)  . 

In  effect,  Letzkus  is  arguing  the  other  side  of  the  issue.  He  states  that 
while  existing  systems  are  forcing  managers  to  concentrate  on  effective¬ 
ness,  they  are  ignoring  efficiency.  Although  starting  with  a  different 
premise,  his  conclusion  agrees  with  Engel.  In  addition,  Letzkus  (1973) 
proposes  the  following  model  of  Air  Force  performance: 

Organizational  performance  =  Mission  efficiency  +  Mission  effectiveness. 

Writing  from  the  comptroller's  viewpoint,  Letzkus  (1973)  uses  the 
terms  costs  and  benefits  rather  than  input  and  output.  Nevertheless,  his 
discussion  of  benefit  determination  provides  insight  into  the  definition 
of  output  in  an  Air  Force  environment,  a  key  issue  in  defining  and  mea¬ 
suring  productivity.  Letzkus  (1973,  p. 25)  discusses  three  levels  of 
output  measures  : 
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1.  Operations  Indicators  -  Operations  indicators  are  measures 

associated  with  the  outputs  of  activities  which  indicate  in 
raw  financial  terms  what  is  produced  for  money  and  effort 
expended.  Largely  workload  and  performance  statistics, 
these  measures  provide  little  insight  into  how  well  needs 
or  objectives  are  being  met.  Operations  indicators  are 
often  selected  on  the  basis  of  simplicity  of  understanding 
and  data  availability  rather  than  on  the  basis  of  relevance 
and  include  the  kind  of  data  commonly  used  for  the  deter¬ 
mination  of  unit  costs. 

With  reference  to  Air  Force  activities,  operations  indicators 
would  include  such  measures  as  hours  or  sorties  flown,  work 
orders  processed,  the  number  of  engines  repaired,  students 
graduated,  hours  or  weeks  of  training,  meals  served,  etc. 
These  measures  indicate  work  performed.  They  do  not,  how¬ 
ever,  indicate  the  quality  of  performance  or  the  results 
achieved. 

2.  Program  Impact  Indicators  -  Program  impact  indicators  are 

directly  related  to  a  public  need  or  policy  and  (theoret¬ 
ically)  are  expressed  in  or  implied  by  stated  program 
objectives.  Outputs  of  programs  should  be  described  in 
terms  that  provide  a  basis  for  evaluating  actual  against 
planned  accomplishments.  These  indicators  generally  are 
relevant  to  levels  above  the  operating  level.  In  the 
context  of  this  definition  of  program  impact  indicators, 
a  relevant  Air  Force  example  would  be  aircraft  accidents 

averted .  adequacy  and  quality  of  repaired  engines 

received  by  operating  activities,  the  tactical  assistance 
resulting  from  ordance  delivered,  etc. 

3.  Social  Indicators  -  Social  indicators  reflect  changes  in 

social  conditions  resulting  from  a  combination  of  programs, 
but  not  solely  attributable  to  any  one  of  them.  National 
defense  is  a  public  good  which  influences  the  "quality  of 
life."  The  problem  is  to  find  a  social  indicator (s)  which 
measures  national  defense. 

These  three  levels  of  indicators  represent  a  spectrum.  As  one 
moves  from  operations  indicators  to  social  indicators,  the  measures 
become  more  relevant  but  correspondingly  more  difficult  to  measure. 

Most  efficiency-based  approaches  to  productivity  are  only  concerned  with 
the  lowest  level  of  this  spectrum,  operations  indicators. 

A  similar  view  of  the  limitations  of  efficiency  based  measures  was 
expressed  by  a  faculty  member  in  an  Air  Force  school. 


Productivity  (as  traditionally  defined)  does  not  indicate 
whether  an  organization  is  doing  its  job.  It  does  not 
reflect  whether  the  organization  is  keeping  current, 
adapting  to  new  procedures,  technology  or  new  situations. 
Therefore,  a  definition  of  productivity  must  include 
effectiveness  as  well  as  workload.  However,  since  goals 
are  not  ususally  defined  we  can't  measure  effectiveness 
therefore  we  fall  back  to  a  dependence  on  workload.  We 
don't  focus  on  quality.  Productivity  should  be  defined 
as  workload  and  goal  accomplishment. 

Another  military  researcher's  disenchantment  with  the  efficiency  de¬ 
finition  of  productivity  was  expressed  as  follows: 

Work  group  effectiveness  is  a  better  term  than 
productivity  defined  as  a  ratio  of  output  to  input 
because  there  are  some  organizations  which  have  re¬ 
stricted  outputs  or  their  output  product  is  readiness. 

In  that  case  a  difference  in  the  inputs  would  change 
the  ratio.  ...  I'm  not  sure  we  can  measure  the 
effectiveness  of  "being  there."  This  is  the  organiza¬ 
tional  level.  But  we  can  get  specific  inputs  and 
outputs  down  at  the  sub-unit  level.  We  often  fail  to 
see  this.  Down  at  the  work  group  level  they  are  pro¬ 
ducing  and  they  produce  measureable  quantities  of  output. 

Still  another  researcher  stated  the  issue  in  this  way: 

Productivity  must  use  or  include  both  efficiency 
and  effectiveness.  In  peacetime  the  focus  is 
on  efficiency  -  in  wartime  effectiveness  is  all 
that  matters. 

Summary  of  Field  View  of  Productivity 

The  prevailing  viewpoint  from  field  managers  and  researchers  in¬ 
terviewed  is  that  the  "official"  Air  Force  definition  of  productivity, 
the  ratio  of  goods  produced  or  services  rendered  to  the  resources 
expended,  is  too  narrow.  Managers  and  researchers  interviewed  agree 
that  it  is  not  enough  to  consider  only  the  efficiency  with  which  work 
is  performed.  Assessment  of  an  organization's  performance  must  also 
take  into  account  the  extent  to  which  an  organization  is  achieving  its 
mission.  Ironically,  the  Department  of  Labor>  which  fosters  the 
efficiency  measurement  system  used  to  generate  National  productivity 
statistics,  sums  up  the  problem  as  follows: 

The  main  difference  between  the  concepts  of 
productivity  and  efficiency  is  that  the  former 
includes  no  evaluation  in  relation  to  some  over¬ 
all  goal.  A  measure  of  productivity  does  not 
indicate  anything  about  the  appropriateness  of 
the  activity  itself.  The  program  or  activity 
and,  consequently,  the  output  is  taken  as  given. 

Thus  interest  only  in  questions  of  productivity 
can  Lesult  in  efficiently  carrying  out  the  wrong 
functions  (U.S.  Department  of  Labor,  1972,  P-  6)  . 
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Productivity  in  Air  Force  Organizations;  A  Proposed  Conceptualization 

An  article  by  Balk  (1975)  provides  a  useful  conceptual  framework  for 
viewing  productivity  in  Air  Force  organizations.  Using  a  very  simple 
model  (Figure  1)  Balk  (1975) explains  that  the  production  process  involves 


Action  Alternatives 


i - 

• 

Input 

i 

Process 

„  Output  v 

1 

* 

T 

1 

V 

t  Standards 

. 7 

o 

•  People 

•  Methods 

•  Units 

■p 

K 

2 

•  Same  Dimen- 

•  Space 

•  Schedules 

•  Events 

sions  As 
Output 

•  Capital 
Assets 

•  Layout 

•  Shape 

•  Costs 

•  Motivation 

•  Timeliness 

•  Management 

•  Satisfaction 

Styles 

Figure  1.  Elements  of  the  Production  Process 
(Adapted  from  Balk,  1975) 


converting  inputs  into  outputs  and  comparing  these  outputs  to  standards 
to  determine  if  the  process  should  continue  or  be  modified.  If  outputs 
fail  to  meet  standards,  one  can  revise  inputs,  change  the  processing 
procedures,  or  revise  the  standards.  Ratios  are  computed  to  analyze 
and  control  this  process.  Comparisons  of  output  to  input  are  known  as 
efficiency  ratios.  Comparisons  of  output  to  standards  are  known  as 
effectiveness  ratios.  As  Balk  (1975)  points  out,  productivity  has  been 
synonomous  with  efficiency. 

Industry  and  classical  economics  have  used  the 
simple  output  to  input  ratio  because  quality  is 
a  step  in  the  manufacturing  process.  In  service 
industries,  customers  are  assumed  to  buy  only 
"satisfactory"  outputs.  These  market  mechanisms 
do  not  operate  in  most  of  the  public  sectors.  Thus 
a  productive  process  in  one  which  optimizes  efficiency 
and  effectiveness  ratios  (1975,  p.  130)  . 

This  leads  Balk  (1975,  p.  130)  to  define  productivity  as  follows: 
Productivity  =  efficiency  +  effectiveness 


or 


Productivity  = 


Output 

Input 


Output 

Standard 


This  definition  of  productivity  suggests  that  efficiency  and 
effectiveness  must  strike  a  balance  in  order  to  optimize  productivity. 
They  are  not  independent  and  Balk  (1975)  acknowledges  that  the  notion 
of  "quality  fights  quantity"  is  included  in  his  conception  of 
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productivity.  He  goes  on  to  argue  that  the  key  to  understanding 
productivity  is  understanding  outputs.  We  have  traditionally  focused 
on  the  physical  characteristics  of  output.  We  have  measured  quantity, 
form,  shape,  adjerence  to  schedule,  etc.  But,  writing  about  the  public 
sector.  Balk,  (1975)  maintains  that  there  are  also  intangible  charac¬ 
teristics  of  output  such  as  client  satisfaction,  value  to  the  community, 
and  other  "feelings"  about  the  results  of  government  services.  This 
"impact"  of  tne  outputs  on  the  environment  is  an  important  productivity 
dimension.  However,  according  to  Balk  (1975) ,  these  less  tangible 
aspects  of  output  make  us  uncomfortable  because  we  don't  know  how  or  when 
to  apply  them. 

As  a  way  out  of  this  dilemma.  Balk  (1975)  postulates  the  concept 
of  "task  ambiguity" .  This  concept  builds  on  the  common  sense  notion 
that  more  routine  work  is  easier  to  measure  than  is  less  routine  work. 

So  the  amount  of  "routine"  in  work  may  determine  how  we  measure  aspects 
of  work  and  how  we  use  the  numbers.  Task  ambiguity  relates  to  factors 
of  work  such  as  variety,  control  over  input,  dependence  on  others,  and 
skill  requirements.  As  variety  increases,  and  as  control  over  input 
decreases,  and  as  dependence  on  others  and  skill  requirements  increase, 
then  task  ambiguity  increases.  Balk  argues  that  task  ambiguity  is 
related  to  styles  of  measurement.  He  hypothesizes  that  as  task  ambiguity 
increases : 

1.  Measurement  reliability  and  validity  decreases 

2.  The  utility  of  efficiency  ratios  decreases 

3.  The  importance  of  effectiveness  ratios  increases 

4.  The  possibility  of  a  single  measure  to  define  a 

productivity  situation  decreases  (Balk,  1975,  p.  131). 

Balk's  assertion  that  the  characteristics  of  tasks  have  an  impact  on  the 
type  of  measurement  is  similar  to  the  previously  discussed  idea  that  the 
dominant  technology  in  an  organization  (e.g.,  long-linked,  mediating, 
intensive)  has  an  effect  on  definition  of  organizational  effectiveness 
(Mahoney  and  Frost,  1974) . 

The  conceptual  scheme  and  line  of  reasoning  proposed  by  Balk  leads 
to  conclusions  that  are  consistent  with  the  views  on  productivity  ex¬ 
pressed  by  Air  Force  managers.  Balk's  model  would  support  the  manager's 
comments  that  efficiency-based  definitions  and  measures  are  too  narrow 
to  adequately  assess  productivity  in  Air  Force  organizations.  This 
reasoning  suggests  that  a  broader  definition  of  productivity  in  Air 
Force  organizations  be  adopted  which  embraces  the  concepts  of  both 
efficiency  and  effectiveness.  Such  a  definition  is: 

Productivity  in  Air  Force  organizations  refers  to  the 
volume  of  resources  used  to  produce  products  and 
services  (efficiency)  and  the  extent  to  which  these 
products  and  services  conform  to  acceptable  standards 
of  mission  performance  (effectiveness) . 

This  definition  implies  that  productivity  should  be  measured  in  Air 
Force  organizations  by  using  a  family  of  efficiency  and  effectiveness 
ratios.  -in 


IV.  PRODUCTIVITY  MEASUREMENT  METHODS:  DISCUSSION  AND  CRITIQUE 


This  chapter  considers  various  approaches  to  measuring  organiza¬ 
tional  productivity.  It  includes  an  introductory  section  which  describes 
a  number  of  basic  considerations  in  organizational  measurement  and  con¬ 
cludes  with  a  discussion  of  the  criteria  that  should  be  used  in  analyzing 
productivity  measurement  approaches.  Following  a  description  of  the 
desirable  features  of  productivity  measures,  individual  methods  are 
described  and  critiqued.  Categories  of  measurement  methods  reviewed 
are  efficiency  measurement,  effectiveness  measurement,  and  integrated 
approaches.  The  chapter  concludes  with  a  discussion  of  the  impact  of 
productivity  measures  on  the  phenomenon  of  productivity  itself. 

General  Measurement  Issues 


Measurement  -  A  Definition 

Measurement  is  a  process  which  involves  assigning  numbers  to  objects, 
events  or  attributes  according  to  specified  rules  or  procedures.  More 
precisely,  a  measurement  operation  "is  a  standardized  rule  that  maps 
each  of  a  set  of  objects  into  one,  and  only  one,  of  a  set  of  categories 
or  numbers"  (Kays,  1967,  p.  5) .  The  concept  of  standardization  is 
critical  to  the  measurement  process  and,  in  practice,  means  that 
different  people  who  apply  the  rule  to  particular  events  and  situations 
obtain  very  similar  results.  Thus,  an  organizatonal  productivity 
measurement  process  is  standardized  if  two  different  analysts  assign  the 
same  values  to  the  same  organizations  at  a  given  point  in  time  (Nunnally, 
1967)  . 

In  the  definition  of  measurement,  it  is  important  to  note  that 
numbers  are  assigned  to  attributes  of  objects  or  events.  "Strictly 
speaking,  one  does  not  measure  objects  -  one  measures  their  attributes" 
(Nunnally,  1967,  p.3) .  The  implicaton  of  this  point  for  organizational 
measurement  is  that  one  or  a  few  measurements  of  an  organization  (e.g., 
size,  productivity,  etc.)  should  not  be  used  to  characterize  or 
"evaluate"  the  (total)  organization.  Many  attributes  must  be  measured 
if  one  is  to  adequately  describe  in  quantitative  terms  an  entity  as 
complex  as  an  organization. 

Benefits  of  Measurement 

Use  of  standardized  organizational  measurement  procedures  offers 
many  benefits  to  both  researchers  and  managers  (Nunnally,  1967) . 

One  benefit  is  objectivity.  Use  of  standardized  measurement  procedures 
takes  much  of  the  guesswork  out  of  observations,  and  allow  independent 
verification  of  organizational  attributes.  Secondly,  numerical  indicies 
provided  by  measurement  procedures  allow  reporting  of  results  in  finer 
detail  and  more  precisely  than  would  be  possible  with  subjective 
descriptions.  These  numerical  indices  also  permit  use  of  mathmatical 
and  statistical  analysis  procedures.  Without  such  analytical 
tools,  organizational  research  and  practice  would  be  seriously 


31 


i 


impaired.  Finally  measurement  results  facilitate  communication  between 
researches  managers,  members  of  the  organization,  and  others  who  in¬ 
fluence  or  are  affected  by  the  organization's  activities.  Unless 
standardized  measurement  procedures  exist,  it  is  virtually  impossible 
to  evaluate  an  organizaton ' s  performance  and  to  communicate  to  an 
interested  audience  that  progress  has  been  made. 

Levels  of  Measurement 

The  type  of  measurement  operation  selected  or  developed  depends 
on  the  attribute  to  be  measured  and  the  purpose  of  measurement. 

Four  levels  of  measurement  are  typically  identified  by  statisticians: 
Nominal,  ordinal,  interval  and  ratio  (Hays,  1976). 

Nominal .  In  nominal  level  measurement,  numbers  are  used  either  as 
labels  (e.g.,  classrooms  in  a  building  are  numbered  1,  2,  3,  etc.) 
or  as  a  means  of  assigning  people,  objects,  or  events  to  categories 
(e.g.,  male  =  1,  female  =  2). 

The  use  of  numbers  in  this  way  does  not  imply  any  quantitative 
meaning  in  terms  of  the  amount  of  an  attribute  (Nunnally,  1967) . 

Ordinal .  In  ordinal  measurement,  (1)  objects  of  measurement  are 
ordered  from  most  to  least  with  respect  to  some  attribute,  (2)  there 
is  no  indication  of  how  much  of  the  attribute  the  measurement 
object  posesses,  and  (3)  there  is  no  indication  of  how  far  apart 
the  objects  are  on  the  attribute  being  measured  (Nunnally,  1967) . 

For  example,  if  military  organizations  are  assigned  ranks  depend¬ 
ing  on  their  level  of  readiness  (category  1,  category  2,  etc.) 
this  would  be  an  example  of  ordinal  measurement.  The  only  con¬ 
clusion  that  can  be  drawn  from  this  classification  scheme  is  that 
category  1  organizations  are  higher  on  the  attribute  (readiness) 
than  category  2  organizations,  and  that  category  2  organizations 
are  higher  than  category  3  organizations,  and  so  on. 

Interval .  Interval  measurement  procedures  lead  to  a  rank  ordering 
of  objects  of  measurement  with  respect  to  an  attribute  when  the 
distances  between  objects  are  known.  However,  ordinal  procedures 
do  not  provide  any  information  about  the  absolute  magnitude  of  the 
attribute  for  any  object  of  measurement  (Nunnally,  1967) .  The 
use  of  a  productivity  index  is  an  example  of  interval  measurement. 
For  example,  if  the  measured  productivity  in  1980  is  selected  as 
the  arbitrary  zero-point  or  base  year,  productivity  in  successive 
years  is  expressed  in  relation  to  the  1980  value.  For  example, 
productivity  values  were  obtained: 


Year 

Output 

Input 

Productivity 

1980 

4 

10 

.40  i 

1981 

6 

12 

.50 

1982 

8 

14 

.57 
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When  converted  to  an  index,  values  for  the  three  years  might  be 
expressed  as:  1980  =  100,  1981  =  125,  and  1982  =  143. 

The  numbers  100,  125  and  143  form  an  interval  scale. 


Ratio  Ratio  measurement  includes  all  the  characteristics  of 

_ _ _ _  a 

interval  measurement,  and  the  quantities  are  expressed  in  relation 
to  an  absolute  zero.  Measures  of  production  constitute  ratio 
measurement  when  physical  units  are  counted.  For  example,  if  the 
number  of  aircraft  refueled  on  five  consecutive  days  were  20,  8,  12, 
7,  and  10, these  numbers  form  a  ratio  scale.  Since  zero  aircraft 
refueled  means  an  absence  of  the  attribute  (number  of  aircraft 
refueled) ,  this  scale  has  a  meaningful  zero  point. 

Importance  of  Level  of  Measurement 

In  developing  and  interpreting  measures  of  productivity,  the  level 
of  measurement  produced  by  a  particular  measurement  operation  is 
significant.  If  a  given  measurement  operation  produces  only  ordinal 
measurement  and  ratio  interpretations  are  attempted,  they  will  lead  to 
inaccurate  conclusions.  In  addition,  the  application  of  statistical 
procedures  to  productivity  indicators  must  be  done  in  accordance  with 
the  level  of  measurement  represented  by  the  indicator.  For  example, 
attempting  to  compute  an  average  (arithmetic  mean)  for  ordinal  data  is 
not  justified. 


Characteristics  of  Good  Productivity  Measures 


In  addition  to  level  of  measurement,  there  are  also  important 
attributes  of  productivity  measures.  Hurst  (1980,  pp.  43-49)  described 
nine  characteristics  which  he  lists  as  desirable  for  measures  of 
organizational  performance. 

1.  Controllable .  The  person  or  group  being  measured  should 

have  control  over  the  aspects  of  performance  measured  by 
the  indicator.  In  other  words,  groups  should  not  be  held 
accountable  for  results  that  are  outside  of  their  control. 
For  example,  percentage  of  on-time  takeoffs  may  not 
accurately  measure  productivity  of  an  Air  Force  operations 
section  since  inputs  from  maintenance  and  supply  are  also 
involved. 

2.  Congruent .  The  performance  measure  for  a  sub-system  should  be 

compatible  with  the  overall  mission  and  objectives  of  the 
larger  organizational  system  of  which  it  is  a  part  .  For 
example,  areas  of  performance  measured  for  a  branch  should 
be  relevant  to  the  overall  mission  of  the  squadron. 

3.  Measurable.  The  characteristic  should  be  quantifiable  through 

procedures  that  are  feasible.  In  addition,  a  measurable 
characteristic  should  be: 


a.  Unequivocal.  Not  subject  to  misinterpretation, 
and  sensitive  to  actual  changes  in  performance 
that  occur; 

b.  Reproducible.  Similar  performance  changes  will 
produce  similar  measures  repeatedly; 

c.  Accurate,  Not  subject  to  random  or  systematic 
basis  that  will  introduce  measurement  error;  and 

d.  Objective .  Multiple  observers  of  the  characteristics 
should  agree  on  what  is  good  performance  and  what 

is  bad. 

4.  Understandable .  The  extent  to  which  the  relationship 

between  level  of  performance  and  the  measure  will  be 
understood  by  the  individual  being  measured  .  For  example, 
complex  schemes  used  to  weight  labor  input  would  be 
less  understandable  than  simple  unweighted  schemes. 

5.  Chops able.  The  extent  to  which  the  people  being  measured 

have  a  say  in  the  measures  by  which  their  organization 
will  be  judged.  For  example,  workload  factors  imposed 
by  regulation  are  less  choosable  than  factors  generated 
at  the  local  level . 

Hurst  (1980,  p.  46)  acknowledges  that  not  all  of  these  attributes  of 
good  measures  can  be  achieved  at  the  same  time.  An  example  of  the  type 
of  trade-offs  often  encountered  is  a  trade-off  between  accuracy  and 
understandability .  For  example,  a  productivity  indicator  in  a  vehicle 
maintenance  organization  may  be: 

number  of  engines  tuned 
number  of  hours  worked 

Since  all  engines  are  not  equivalent,  it  may  be  necessary  to  develop  a 
"standard  output  unit."  For  example,  if  the  shop  repairs  1/4  ton 
vehicles,  3/4  ton  vehicles,  and  2-1/2  ton  vehicles,  the  output  unit 
may  be  the  number  of  3/4  ton  engine  equivalents  repaired.  While  this 
would  more  accurately  reflect  output,  it  would  be  less  understandable 
to  members  of  the  organization  whose  performance  is  being  measured 
than  simply  counting  the  number  of  engines  repaired. 

Other  common  tradeoffs  discussed  by  Hurst  (1980)  include:  Congruence 
versus  Measurability,  Aggregation  to  higher  organizational  levels  versus 
Congruence,  and  Motivation  versus  Interference.  This  last  issue,  the 
impact  of  measurement  on  the  phenomenon  being  measured,  will  be  discussed 
later  in  this  chapter. 

A  recent  study  (A.T.  Kearney,  Inc.,  1978)  of  productivity  in  physical 
distribution  outlined  a  somewhat  different  list  of  desirable  characteristics 
of  productivity  measures.  A.T.  Kearney,  Inc. (1978,  pp.  11-13)  identified 
seven  criteria  for  judging  productivity  measures: 
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1.  Validity ,  A  valid  measure  is  one  that  accurately  reflects 

changes  in  real  productivity.  In  a  receiving  operation 
that  involves  moving  palletized  goods  from  a  receiving 
dock  to  the  warehouse,,  the  number  of  pallet  loads  per 
hour  would  be  a  valid  measure.  Cases  per  hour  or  pounds 
per  hour  would  be  a  less  valid  measure  due  to  the  fact 
that  cargo  density  may  vary. 

2.  Coverage  The  more  completely  the  output  accounts  for  all 

uses  of  an  input  resource,  the  more  accurately  that  input 
can  be  tracked.  In  the  example  above,  if  the  operator 
puts  cargo  on  shelves  and  loads  and  unloads  cargo  with  a 
handtruck  as  well  as  operates  the  fork  lift,  then  it  may 
be  necessary  to  have  three  output  measures  in  order  to 
account  for  the  individual's  work  time. 

3.  Comparabil ity .  In  order  for  measures  to  be  compared  across 

time  they  must  be  reduced  to  a  common  denominator.  One 
way  to  do  this  is  to  multiply  the  number  of  units  produced 
by  the  standard  times  allowed  for  the  task.  If  standards 
do  not  exist  then  "equivalent  units"  can  be  defined  and 
used  to  aggregate  outputs. 

4  Completeness .  All  significant  inputs  should  be 

included  in  the  measure.  In  the  warehouse  example, 
potential  inputs  would  be  labor,  energy,  equipment  de¬ 
preciation,  floorspace,  etc. 

5.  Usefulness .  To  be  useful  to  a  manager,  a  productivity  measure 

should  guide  that  manager  toward  some  effective  action. 

6.  Compatibility .  Whenever  possible,  data  for  the  measure  should 

be  available  from  existing  sources  of  data  and  management 
information  systems. 

7.  Cost  Effectiveness.  The  benefits  gained  from  a  measure  must 

exceed  the  costs  associated  with  collecting  the  data. 

Some  of  the  attributes  proposed  by  A.T.  Kearney,  Inc.,  (1978)  are 
basically  the  same  as  those  proposed  by  Hurst  (1980) ,  although  the 
Kearney  list  emphasizes  feasibility  and  cost  to  a  greater  degree. 
Nevertheless,  these  two  lists  provide  a  comprehensive  set  of  criteria 
against  which  to  judge  the  adequacy  of  productivity  measures. 

Review  and  Critique  of  Productivity  Measures 

A  scheme  for  organizing  productivity  measurement  approaches  is 
presented  in  Figure  2.  This  simple  taxonomy  forms  the  organizational 
scheme  used  in  the  following  discussion  . 
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Fiqurs  2.  Taxonomy  of  Productivity  Measurement  Methods 


Efficiency  Measurement 


Output/Input  Measures.  The  first  category  of  efficiency  measures 
presented  in  Figure  2  includes  measures  which  define  efficiency  as  a 
ratio  of  outputs  to  inputs.  Both  outputs  and  inputs  can  be  expressed  in 
terms  of  physical  units  (e.g.,  pounds,  hours,  miles,  gallons,  number 
of  items  etc.)  or  in  terms  of  cost  or  value  expressed  in  dollars.  Using 
these  dimensions,  the  four  resulting  types  of  output/input  efficiency 
ratios  are  depicted  in  Figure  3. 


Outputs 


Physical  Quantity 


Price 


Physical 

Quantity 


Form  I 


Inputs 


Form  II 


Price  Form  III 


Form  IV 


Figure  3.  Types  of  Efficiency  Ratios 


These  four  types  are: 

1 .  Output  in  Physical  Quantity 
Input  in  Physical  Quantity 

2 .  Output  in  Price  Form 
Input  in  Physical  Quantity 

3 .  Output  in  Physical  Quantity 
Input  in  Price  Form 

4.  Output  in  Price  Form 
Input  in  Price  Form 


(Form  I) 

(Form  II) 

(Form  III) 

(Form  IV) 


Two  additional  dimensions  for  classifying  efficiency  measures  are  level 
(micro  versus  macro)  and  type  (partial  versus  total  factor).  In  this 
discussion,  micro  level  efficiency  indicators  are  used  to  measure 
efficiency  at  the  work  group,  work  center,  branch,  or  squadron  levels. 
Measures  applied  to  organizations  larger  than  squadron  level  are  termed 
"macro".  Measurement  type  refers  to  whether  or  not  the  denominator  of 
the  ratio  includes  a  single  input  (partial)  or  multiple  inputs  (total 
factor).  Using  all  factors  to  classify  efficiency  measures  would  lead  to 
the  taxonomy  shown  in  Figure  4. 
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Level 


Macro 


Micro 


Figure  4.  Taxonomy  of  Efficiency  Measures 

The  discussion  of  efficiency  measures  will  not  treat  all  the 
possible  measures  corresponding  to  the  cells  of  Figure  4.  The  discussion 
will  focus  on  the  four  types  of  output/input  measures  defined  in  Figure 
3.  Only  Type  1  will  be  broken  down  into  macro  and  micro  levels.  How¬ 
ever,  before  discussing  each  of  the  four  ratio  forms,  several  overall 
considerations  in  evaluating  efficiency  measures  should  be  discussed. 

With  respect  to  level  of  measurement,  macro  and  micro  indicators  have 
different  purposes.  Macro  measures  (e.g.,  Command-wide  or  Air  Force¬ 
wide)  are  useful  at  the  policy  level  and  are  useful  for  budget  for¬ 
mulation  but  have  little  value  for  the  operating  level.  Even  though  the 
data  for  macro  level  measures  are  normally  aggregated  up  from  lower 
levels,  the  "gross"  measures  of  output  used  are  not  meaningful  to 
managers  who  want  to  analyze  the  sources  of  productivity  problems  in 
their  organizations.  Many  of  the  output  indicators  used  in  the  pro¬ 
ductivity  reporting  system  are  workload  measures  (e.g.,  number  of  active 
duty  pay  accounts  serviced,  authorized  base  population  served,  weighted 
population  supported,  etc.).  Managers  who  input  data  normally  do  not 
receive  feedback  about  their  organization's  productivity.  Line  managers 
do  not  know  productivity  levels  because  the  output  data  are  combined 
with  input  (usually  only  labor)  data  at  organizational  levels  above  the 
work  center  or  branch. 

In  general,  total  factor  measures  are  preferable  to  partial 
measures  because  they  provide  a  more  accurate  accounting  of  an  organ¬ 
ization's  "true"  efficiency.  This  is  particularly  true  for  higher  level 
organizations.  A  rule  of  the  thumb  for  determining  the  inputs  for 
efficiency  ratios  is  that  the  efficiency  measure  for  a  unit  should  in¬ 
clude  in  the  denominator  only  those  inputs  over  which  the  organization 
has  some  control  (Thor,  1979) . 

One  of  the  difficulties  in  using  efficiency  ratios  in  the  Air 
Force  is  that  managers  have  very  little  control  over  the  inputs  to  the 
process.  They  cannot,  at  least  over  the  shortrun  adjust  input  levels  of 
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labor  to  the  workload.  Organizations  are  authorized  and  assigned  per¬ 
sonnel  in  relation  to  their  workload.  Therefore,  there  are  a  given 
number  of  "hours"  of  input  available  to  a  unit  commander.  While  these 
hours  can  be  allocated  to  tasks  in  different  ways,  the  total  labor  in¬ 
put  is,  for  all  practical  purposes,  fixed.  Thus,  the  only  variable 
is  workload.  Organizations  vary  in  terms  of  the  amount  of  control  they 
have  over  their  workload.  Where  there  are  slack  periods  and  labor  in¬ 
put  is  fixed,  the  labor  efficiency  is  poor.  Where  workload  is  heavy, 
labor  efficiency  increases.  Neither  high  nor  low  efficiency  value  is  a 
valid  measure  of  the  efficiency  of  such  organizations.  The  efficiency 
ratio  simply  measures  fluctuations  in  workload  which  are  outside  the 
organization's  control. 

A  final  issue  with  efficiency  measures  concerns  the  desirable 
characteristics  of  output  indicators.  In  its  guide  to  implementing 
productivity  programs  in  the  Federal  Government,  the  Joint  Financial 
Management  Improvement  Program  (1977)  described  some  "tests"  that  may 
be  applied  in  selecting  or  defining  outputs: 

1 .  Mutually  exclusive.  Can  the  input  required  to  produce  the 

output  be  readily  identified? 

2.  Process  definable.  Are  the  same  steps  required  to  complete 

the  operation  each  time? 

3.  Countable.  Can  the  number  of  units  produced  or  services 

provided  be  quantified? 

4.  Uniform  over  time.  Will  the  nature  of  the  product  or  service 

remain  relatively  stable  over  a  reasonable  period  of  time? 

5.  Mission  oriented  .  Does  the  product  or  service  represent  all 

or  a  significant  part  of  the  mission  of  the  activity 
being  measured? 

6.  Quality  definable.  Is  quality  of  the  product  relatively  stable, 

or  at  least  definable?  Quality  becomes  a  problem  only 
when  it  changes,  but  definition  is  needed  to  determine  if 
a  change  has  occured.  If  changes  have  occurred,  these  can 
usually  be  factored  in  to  adjust  the  productivity  equation. 

7.  Data  readily  available.  To  what  extent  is  data  available  from 

existing  systems? 

8.  Directness  of  the  measure.  Are  the  measures  direct  or,  if 

proxy  (indirect)  measures  are  necessary,  is  there  a 
rational  relationship  between  the  output  and  the  measure 
( JFMIP  ,  1977,  p. 13-14)  . 

With  these  general  considerations  in  mind,  this  section  will  discuss 
each  of  the  four  forms  of  efficiency  ratios. 
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Form  I  -  Physical  Quantity  Input  and  Output  -  Micro  Level 

a.  Description.  There  are  many  examples  of  efficiency  indices 
that  involve  physical  quantities.  Two  examples  are 
presented  below: 

Labor  partial  =  Number  of  line  items  issued  by  supply  branch 
Total  hours  worked 

Energy  partial  =  Number  of  passenger-miles  driven 
Number  of  gallons  of  fuel  used 

b.  Critique.  Physical  quantity  measures  offer  many  advantages 
for  efficiency  measurement.  Since  they  are  not  affected 
by  inflation,  they  can  be  compared  directly  with  data  from 
previous  periods.  Other  advantages  include  relative  ease 
of  computation,  ease  of  understanding,  and  generally 

high  acceptability.  Disadvantages  of  the  measure  in  an 
Air  Force  organization  are  that  the  index  may  fluctuate 
as  a  result  of  factors  that  are  not  controllable  by 
the  organization.  When  managers  have  little  control  over 
the  number  of  personnel  assigned,  as  is  the  case  in  the 
Air  Force,  labor  partial s,  such  as  the  first  example 
above  are  misleading.  In  addition,  aggregating  outputs, 
such  as  line  items,  can  be  misleading  if  there  are  large 
differences  in  the  time  to  process  them.  Because  of  this 
problem,  the  Air  Force  productivity  program  makes  use  of 
weighted  line  items  processed  as  an  output  of  supply 
organizations.  In  high  volume  organizations, it  is  a 
reasonable  assumption  that  these  differences  in  un¬ 
weighted  line  items  will  average  out,  but  in  low  volume 
operations  outputs  should  be  standardized  or  weighted 
before  they  can  be  meaningfully  aggregated.  Unless  this 
is  done,  the  indicator  will  have  low  validity  and  low 
acceptability  to  managers.  It  will  also  have  low  re¬ 
levance  as  a  criterion  for  evaluating  organizational 
change  efforts.  Furthermore,  if  the  organization  has  a 
fluctuating  workload  with  idle  periods,  this  will  distort 
the  labor  partial  indicator.  A  further  disadvantage  of 
partial  efficiency  indicators  is  that  they  are  distorted 
by  factor  substitutions  (e.g.,  capital  for  labor). 

Labor  productivity  may  show  a  spurious  improvement  when 
the  source  of  improvement  is  really  a  new  piece  of 
equipment. 

Form  I  -  Physical  Quantity  Input  and  Output  -  Macro  Level 

a.  Description.  Form  I  indicators  are  the  types  of  measures 
typically  reported  by  Air  Force  Headquarters,  DoD  and  the 
Bureau  of  Labor  Statistics.  Generally  these  measures  are 
reported  in  index  terms.  Table  2  provides  an  example  of 
how  such  an  index  is  calculated.  The  data  presented  are 
illustrative  only. 


Table  2 

Computation  of  Labor  Partial  Index 


(a) 

Month  1 
(Base) 

(b) 

Month  2 

(c) 

Index 

(bra)xlOO 

1. 

Output  (Meals  served  in  DoD 

dining  halls) 

1,000,000 

1,200,000 

120 

2. 

Input  (Hours  worked  by  dining 
personnel) 

250,000 

250,000 

100 

3. 

Efficiency  (lf2) 

4 

4.8 

120 

b.  Critique.  Such  macro  index  measures  provide  top-level 
policy  makers  with  information  that  is  useful  for 
budgeting  and  planning  purposes.  The  measures  also 
provide  an  indication,  if  interpreted  with  caution,  of 
the  extent  to  which  labor  resources  are  efficiently 
utilized  in  relation  to  a  previous  period.  There  are 
many  disadvantages  of  the  macro  measures.  Aggregating 
outputs  at  this  level  produces  a  number  which  has 
dubious  meaning  unless  the  outputs  are  extremely 
homogeneous,  are  standardized,  or  are  weighted  before 
combining.  Because  of  the  degree  of  aggregation  required, 
the  measure  is  insensitive  to  small  or  moderate  pro¬ 
ductivity  changes  in  a  given  organization.  While  the 
macro  measures  may  have  broad  policy  meaning,  they  have 
little,  if  any,  benefit  for  the  operating  manager  at  the 
micro  level.  Such  macro  measures  are  difficult  to 
interpret  and  have  little  value  in  evaluating  organiza¬ 
tional  change  programs. 

3-  Form  II  -  Price  Outputs  and  Physical  Quantity  Inputs 

a*  Description .  Form  II  measures  present  outputs  in  terms  of 
dollar  values.  Such  measures  are  widely  used  in  the 
private  sector  where  sales  in  dollars  is  a  frequently 
used  output.  Government  organizations,  including  the 
Air  Force,  have  difficulty  establishing  dollar  values  for 
their  outputs.  As  a  result,  this  form  is  not  widely  used 
in  the  Air  Force. 

An  example  of  a  Form  II  index  is: 

Shipping  value  per  hours  worked  =  value  of  items  shipped 

Labor  hours 


or 
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Utilization  of  space 


Sales  in  dollars 

Square  feet  of  retail  space 


An  indicator  used  in  military  commissaries  is: 


Commissary  efficiency 


Deflated  dollar  value  of  sales 
Commissary  labor  hours 


b.  Critique.  Form  II  indices  are  understandable  and 

acceptable  to  managers  and  are  easy  to  compute  since  most 
organizations  maintain  the  necessary  data.  In  their 
"raw"  form,  however ,  these  indices  can  be  misleading.  In 
general,  physical  measures  of  output  are  preferable  to 
output  stated  in  dollars  because  dollar  measures  are 
affected  by  inflation.  Selling  prices  are  affected  by 
wholesale  prices,  selling  expenses,  fixed  asset  costs, 
markups,  etc.  Dollar  values,  even  when  adjusted  for  in¬ 
flation,  do  not  provide  an  accurate  way  to  aggregate 
different  types  of  outputs.  For  example,  in  the  previous 
shipping  example,  unless  items  were  priced  according  to 
the  required  handling  time,  the  output  measure  would 
have  low  validity.  If  separate  indices  were  computed  for 
each  product  and  these  were  adjusted  for  inflation,  then 
the  Form  II  indices  would  be  valid  measures  that  would 
be  useful  for  evaluation  purposes. 


In  a  manufacturing  context,  the  confounding  effects  of 
other  costs  are  often  removed  by  subtracting  the  cost  of 
materials  from  the  value  of  the  output.  This  adjustment 
produces  an  output  value  called  "value  added  by  manufac¬ 
turing."  The  most  accurate  approach  to  adjustment  is  the 
double  inflation  approach  (Greenberg,  1973)  in  which  the 
deflated  cost  of  materials  is  subtracted  from  deflated 
value  of  output.  This  produces  an  index: 


Value  added  per  manufacturing  hour 


Value  added 
Labor  Hours 


In  the  military  environment,  except  for  industrially 
funded  activities,  there  are  relatively  few  situations 
in  which  sales  or  value  added  are  appropriate 
output  measures.  Furthermore,  when  aggregated  output 
measures  are  used  they  are  relatively  insensitive  to 
changes  in  organizational  efficiency  which  may  be  less 
than  the  fluctuations  in  the  dollar  values  because  of 
factors  unrelated  to  organizational  efficiency.  Because 
of  the  need  to  correct  for  inflation  and  other  cost 
factors,  this  measurement  form  requires  considerable 
computation.  This  difficulty,  coupled  with  its  low 
validity,  reduces  its  acceptability  to  managers.  As  a 
result,  it  is  of  little  use  in  guiding  managerial 
decisions.  The  Form  II  measure  is  also  of  little  value 
in  evaluating  the  impact  of  organizational  change  programs 


'I .  Form  III  -  Physical  Quantity  Outputs  and  Price  Inputs 


a.  Description.  Form  III  measures  report  outputs  in  physical 
forms  and  inputs  in  price  or  value  terms  usually  ex¬ 
pressed  in  dollars.  Felsinger  (1976)  provides  an  example 
of  such  a  measure: 


„  .  .  .  .  Number  of  planned  maintenance  actions  accomplished 

Dollar  value  of  materials  consumed  for  repair  parts 

This  index  has  been  proposed  for  use  on  Navy  ships  to 
measure  the  number  of  planned  maintenance  actions  per 
OPTAR  dollar  spent  (Felsinger  1976) .  OPTAR  costs  are 
expenditures  approved  by  the  ship’s  commanding  officer  for 
day-to-day  equippage,  repair  parts,  and  other  operating 
expenses.  Fuel,  utilities,  shipyard  repairs,  and 
personnel  costs  are  excluded.  An  Air  Force  example  from 
the  supply  area  is: 


_  .  ,  _  ...  .  Number  of  weighted  line  items  processed 

Total  factor  efficiency  =  ------  - -2 — — — - - — - — 

Deflated  costs  of  labor,  material,  and  energy 


This  example  makes  use  of  a  weighting  procedure  for  out¬ 
puts.  While  one  might  consider  line  items  processed  to  be 
homogeneous,  in  fact,  some  line  items  take  longer  to 
process  than  others.  There  are  various  procedures  for 
establishing  output  weights  (estimation,  work  measure¬ 
ment,  etc.).  Regardless  of  the  procedure  used,  each 
line  item  is  given  its  appropriate  weight  prior  to 
summing.  For  example,  assume  that  it  takes  a  supply 
clerk  10  minutes  to  issue  a  uniform,  5  minutes  to  issue 
a  weapon  and  1  minute  to  issue  a  tool  kit.  If  weapon 
issues  are  taken  as  the  standard,  then  uniform  issues 
would  receive  a  weight  of  2,  weapons  would  receive  weight 
of  1,  and  tool  kits  would  receive  a  weight  of  .2.  Total 
output  would  be  obtained  as  follows: 


Output  =  2  (Number  of  uniforms)  +  Number  of  weapons  +  (.2)  Number  of  tool  kits 

With  respect  to  inputs,  the  supply  index  includes  all  the 
relevant  cost  factors  involved  in  processing  line  items. 

Total  labor  cost  includes  direct  labor,  indirect  labor 
and  non-available  labor. 


b.  Critique.  Particularly  for  the  military,  where  outputs 
can  be  more  easily  expressed  in  physical  quantity  terms 
than  in  price  terms,  this  Form  III  index  is  very  useful. 
Since  outputs  are  expressed  in  physical  terms, this 
measure  is  easy  to  interpret  and  meaningful  to  managers. 
Its  inverse  can  be  interpreted  as  the  cost  per  unit  of 
output.  With  the  denominator  (inputs)  expressed  in 
terms  of  deflated  costs,  it  is  usually  possible  to  obtain 
the  necessary  input  data  directly  from  existing  cost 
accounting  systems.  The  outputs  expressed  in  physical 
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quantities  are  most  useful  if  they  meet  the  criteria 
for  defining  outputs  presented  earlier.  Form  III 
measures  are  amenable  to  total  factor  productivity 
measurement  as  previously  illustrated  in  the  supply 
example.  Disadvantages  of  Form  III  measures  are  few, 
except  for  organizations  having  a  large  number  of  out¬ 
puts.  Combining  all  outputs  into  a  single  index  may  be 
difficult  from  the  standpoint  of  developing  an  accept¬ 
able  weighting  scheme.  It  may  also  be  undesirable  be¬ 
cause,  as  the  complexity  increases,  its  acceptance  and 
utility  to  managers  decreases.  In  such  an  organization, 
multiple  indices  covering  the  significant  outputs  and 
their  associated  costs  are  recommended. 


5 .  Form  IV  -  Outputs  and  Inputs  in  Price  Terms 

a.  Description .  Form  IV  measures  are  in  essence  financial 
ratios.  Examples  of  Form  IV  efficiency  measures  in¬ 
clude  : 


Labor  efficiency 


Capital  efficiency 


Sales  or  value  added 
Dollars  of  payroll 

Sales  or  value  added 
Dollars  of  capital  invested 


b.  Critique,  Form  IV  ratios  probably  represent  the  most 

common  ratios  used  by  managers;  thus,  they  must  have  high 
acceptability.  However,  these  ratios  are  seriously 
flawed  from  the  standpoint  of  providing  useful  information 
to  managers  about  the  "true"  efficiency  of  the  organization. 
As  has  been  previously  pointed  out,  price  forms  are  less 
useful  than  physical  quantity  formulations.  In  the 
military  context,  this  is  even  more  true.  Even  when 
corrected  for  the  effects  of  inflation  and  expressed  in 
index  terms,  price  or  economic  value  is  not  a  good  basis 
for  aggregating  outputs  in  forming  efficiency  ratios 
unless  prices  are  proportional  to  units  of  work  (labor 
energy,  etc.)  used  to  produce  them.  This  rationale  is 
based  on  Bureau  of  Labor  Statistics  practice.  When  forced 
to  use  unit  price  weights  instead  of  unit  labor  weights 
in  aggregating  output,  the  Bureau  of  Labor  Statistics 
makes  the  assumption  that  unit  value  weights  are  pro¬ 
portional  to  number  of  unit  employee  hours  required  to 
produce  the  outputs  (Bureau  of  Labor  Statistics,  1978) . 

At  the  micro  level.  Form  IV  indices  can  be  used  in 
evaluation  as  an  overall  summary  measure  for  the  branch, 
squadron,  or  plant.  However,  to  be  useful  in  guiding 
management  practice,  they  should  be  supplemented  by 
more  specific  Form  I  or  Form  III  product  or  service 
oriented  indices.  Form  IV  indices  have  little  benefit 
for  purposes  of  evaluating  organizational  change  programs. 
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Performance  Efficiency  Measures.  Performance  efficiency  measures 
are  measures  that  deal  with  the  relationship  between  an  actual  or 
obtained  level  of  performance  and  a  standard  or  expected  level. 

1 .  Performance  efficiency  based  on  engineered  standards 

a.  Description .  Two  types  of  engineered  standard  are  in  use 
in  the  Air  Force  and  in  other  organizations.  One  is  the 
end-item  standard  that  provides  precise  estimates  of 
"should  take"  times  for  an  average  qualified  worker  to 
perform  a  task  (e.g.,  repairing  a  carburetor  on  a 
vehicle  or  processing  a  travel  claim) .  A  second  type  of 
standard  is  the  manpower  standard  used  for  determining 
staffing  lines  and  provides  a  basis  for  computing  the 
number  of  personnel  required  to  handle  a  given  volume 
of  work.  While  both  types  of  standards  can  be  used  to 
generate  measures  of  performance  efficiency,  end  item 
standards  are  most  useful  for  efficiency  indices  at 
the  micro  or  work  center  level .  This  discussion  will 
not  focus  on  the  techniques  of  developing  standards  as 
this  is  a  highly  complex  and  technical  subject.  Instead, 
the  present  discussion  will  focus  on  the  use  of  standards 
in  computing  efficiency  ratios. 


The  Air  Force  activity  which  makes  the  most  use  of  end- 
item  standards  for  work  is  Air  Force  maintenance. 

Standards  have  been  developed  for  most  maintenance  actions. 
For  a  specified  repair,  inspection,  or  other  maintenance 
action,  there  is  an  average  time  that  is  required  for  the 
average  qualified  technician  to  make  a  given  repair.  If 
the  actual  time  for  the  repair  is  compared  with  the 
standard,  this  leads  to  a  measure  of  performance  effi¬ 
ciency.  The  general  form  of  a  performance  efficiency 
index  is: 


Performance  efficiency  = 


Standard  time 
Actual  time 


The  index  shown  above  can  be  used  for  assessing  the 
efficiency  of  an  individual  on  one  task  or  it  can  be 
averaged  to  assess  the  efficiency  of  an  organization. 

A  more  common  way  of  summarizing  such  data  is  in  terms  of 
earned  hours.  Consider  the  example  in  Figure  5. 


b.  Critique.  The  advantages  of  performance  efficiency  measures 
include  ouanti f i ahit i tv ,  understandability ,  and  utility 
to  managers  as  a  guide  to  management  practice.  Assuming 
that  the  standard  is  accurate,  the  process  is  objective 
and  requires  relatively  simole  bookkeeping  and  computa- 
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Task:  Processing  adjustments  to  travel  claims  in  a  finance  and 
accounting  office 

Output  measure :  number  of  adjustments  processed 
Standard:  1.619877  hours  to  process  1  adjustment 
Number  of  work  units  completed:  494 
Actual  hours  expended:  1207 

Performance  efficiency:  Work  units  completed  x  Standard 

Actual  hours 


Earned  hours 
=  Actual  hours 


=  494  x  1.619877 

1207 


66% 


Figure  5.  Illustration  of  Performance  Efficiency  Computation 
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tional  procedures.  As  an  evaluation  tool,  the  efficiency 
index  provides  an  acceptable  criterion  assuming  both  the 
input  (labor  hours)  and  the  workload  are  under  the  control 
of  the  organization.  For  example,  if  an  organization 
has  a  fluctuating  workload  generated  by  an  outside 
organization  and  it  cannot  adjust  input  accordingly,  its 
rate  of  efficiency  will  not  be  a  useful  gauge  of  the 
organization's  actual  efficiency. 

A  disadvantage  of  the  approach  is  that  development  of 
engineered  standards  is  expensive.  Standards  which  are 
not  updated  as  work  and  work  processes  change  lose  their 
applicability  over  time.  There  is  usually  some  resistance 
by  workers  to  this  procedure  unless  they  are  consulted 
and  have  an  input  into  the  standards  development  process. 
This  and  other  efficiency-based  approaches  can  be  criti¬ 
cized  if  they  make  no  provision  for  changes  in  quality 
of  the  output.  The  approach  assumes  that  quality  is 
constant  for  the  units  of  work  counted.  In  high  volume 
operations,  where  work  is  quite  standardized  and  other 
quality  monitoring  procedures  are  in  effect,  this  assump¬ 
tion  may  be  reasonable.  However,  in  less  routine  work 
centers,  where  there  are  some  decisions  to  be  made  by 
workers  and  some  discretion  in  processing  is  required, 
this  approach  is  expensive  to  apply  and  maintain  and 
probably  not  cost-effective.  Performance  efficiency 
indices  using  engineered  standards  are  useful  in  situations 
that  fit  the  constraints  underlying  the  method.  Most  of 
the  problems  with  the  method  result  from  poor  implementa¬ 
tion  and  failure  to  keep  standards  current  rather  than 
from  flaws  of  the  methodology  itself. 

2 .  Performance  efficiency  based  on  statistical  standards . 

a.  Description.  The  illustration  presented  in  Figure  5  deals 
with  a  single  workload  factor.  In  the  development  and 
application  of  manpower  standards  across  work  centers  or 
across  time  periods  for  a  given  work  center,  multiple 
workload  factors  are  typically  utilized.  This  enables  the 
procedure  to  comprehensively  cover  the  organization's 
workload.  For  example,  the  work  outputs  for  an  office 
of  administrative  services  within  a  Federal  Agency  is 
measured  on  four  outputs:  number  of  pieces  of  corre¬ 
spondence  routed,  number  of  pieces  of  cash  mail  handled, 
number  of  transactions  in  an  internal  fund,  and  number  of 
travel  inquiries  answered.  Inputs  are  the  number  of  hours 
worked.  Using  historical  data  from  a  period  of  52  weeks, 
a  multiple  regression  analysis  is  conducted  relating 
outputs  to  input.  This  process  produces  an  equation 
that  can  be  used  to  predict  hours  worked  on  the  basis 
of  variations  in  the  level  of  output. 
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An  example  of  such  an  equation  is: 

Total  hours  =  39.92  (constant) 

+  .0076  X  No.  pcs.  correspondence 

+  .1245  X  No.  pcs.  cash  mail 

+  .1958  X  No.  fund  transactions 

+  .0449  X  No.  travel  transactions 

Applying  the  equation  to  actual  data  for  a  week  yielded 
the  following  results  (U.  S.  Department  of  Commerce, 
undated) : 


Total  Standard  Hours  earned  =  39.9271  +  8868  x  .0076 

+  354  x  .1245 
+  153  x  .1958 
+  425  x  .0449 


=191.9368 


Productivity  Index  = 


Total  std.  hours  earned 
Hours  worked 


x  100 


191.9368  x  10o 
128 


=  1.50  x  100  =  150 


Thus,  the  productivity  for  the  week  was  150%  compared  to 
the  base  period,  the  average  for  the  52  previous  weeks. 

b.  Critique.  This  procedure  offers  a  useful  way  to  track  an 
organization's  labor  efficiency  over  time.  It  requires 
some  statistical  knowledge  to  understand  and  apply  the 
process.  The  output  measure  is  relatively  straightfor¬ 
ward,  and  the  procedure  is  attractive  because  it 
provides  a  mechanism  for  aggregating  different  outputs  at 
the  work  center  level. 


A  disadvantage  of  the  approach  is  that  it  is  useful  only 
for  assessing  labor  productivity.  However,  the  approach 
can  be  used  to  generate  standard  labor  costs  for  outputs 
that  might  be  combined  with  other  input  cost  data  in 
developing  total  factor  measures  for  different  output 
classes.  Acceptance  of  the  approach  by  managers  may  be 
complicated  by  its  statistical  foundation.  Finally,  the 
process  loses  its  effectiveness  quickly  if  the  work 
performed  by  the  organization  changes  in  significant  ways. 
It  appears  to  be  best  suited  to  bureaucratic  organizations 
where  there  is  high  task  specialization  and  where  work 
roles  and  the  actual  work  performed  are  relatively  con¬ 
sistent  over  time. 
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As  an  evaluation  tool,  the  methodology  offers  a  very 
effective  and  objective  way  to  evaluate  the  impact 
of  organizational  change  programs.  If  an  organizational 
change  program  leads  o  improved  efficiency,  this  will  be 
reflected  in  an  increase  in  the  number  of  earned  hours, 
assuming  that  the  volume  of  work  is  elastic  and  can  ex¬ 
pand  with  the  increase  in  organizational  capacity.  In 
addition,  improved  efficiency  will  be  reflected  in 
changes  in  the  regression  weights  when  the  equation  is 
reapplied.  This  could  produce  a  statistical  test  for 
assessing  the  impact  of  organizational  change. 

Performance  Efficiency  Based  on  Historical  or  Technical  Audit 
Standards. 

a.  Description.  Situations  in  which  the  work  load  is  rel¬ 
atively  low  volume,  non-standard  in  nature,  and  requires 
considerable  judgement  to  process  do  not  lend  themselves 
to  either  engineered  or  statistical  standards  of  the  type 
previously  discussed.  The  approach  required  to  establish 
standards  in  such  a  work  center  may  make  use  of  either 
historical  standards  or  what  is  known  as  technical 
estimation.  To  illustrate  the  methodology,  an  example 
will  be  drawn  from  an  Air  Force  procurement  organization. 

Procurement  personnel  process  a  number  of  different  types 
of  procurement  actions.  Each  of  these  actions  varies  in 
terms  of  the  length  of  time  for  processing  and  each  is 
subject  to  various  complexity  factors  which  may  or  may 
not  occur  but  which  have  time  demands.  This  situation 
can  be  depicted  in  the  form  of  a  matrix  as  shown  in  Table  3 . 
Times  within  the  cells  were  established  by  technical 
estimation  using  knowledgeable  procurement  specialists. 

For  each  type  of  procurement  item,  standard  times  for 
processing  can  be  determined  simply  by  adding  the  time 
across  the  complexity  elements  that  apply.  The  time 
estimates  are  based  on  a  combination  of  operational  audits, 
historical  records  and  technical  judgement.  Self-report¬ 
ing  by  the  procurement  officer  provides  the  input  for 
hours  worked  as  well  as  a  categorization  of  the  type  of 
procurement  action  and  its  complexity  factors.  The 
procurement  file  represents  a  clear  audit  trail  and  can  be 
reviewed  as  a  quality  check  to  determine  if  individuals 
accurately  report  time  to  complete  procurement  actions 
and  work  performed. 
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Table  3 

Procurement  Actions,  Complexity  Factors  and  Time  Estimates 


Complexity  Factors 


Types  of 
Procurement 

Actions 

Award  In 
Simplest 
Form 

Sole 

Source 

Small  Bust 
Coordina¬ 
tion 

Mistake 

In 

Bid 

EEOb 

Comp. 

etc. 

Purchase  Order 

3.47 

.40 

.41 

- 

- 

- 

Blanket  Purchase 
Agreement  Order 

3.63 

.40 

.41 

— 

_ 

_ 

Basic  Ordering 
Agreement 

18.94 

_ 

_ 

2.79 

_ 

Restricted, 
Advertised  100% 
Set  Aside 

4.51 

15.32 

2.79 

Change  Order 
Issuance 

14.95 

.41 

_  . 

2.79 

* 

etc. 

- 

- 

- 

- 

- 

- 

^  Business 

Equal  Employment  Opportunity 
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On  a  monthly  basis,  effiency  measures  are  computed  by 
establishing  a  ratio  of  the  standard  time  (earned  hours) 
obtained  from  the  chart  and  actual  hours. 


b.  Critique.  The  primary  advantage  of  this  approach  is  that 
it  can  be  applied  to  work  situations  not  suited  for 
use  of  statistical  or  engineered  standards.  The  logic 
of  the  approach  is  clear  and  straightforward,  thereby 
increasing  its  acceptance  to  management.  A  by-product 
of  the  approach  is  that  it  provides  careful  analysis  of 
the  type  of  work  performed  in  the  work  center.  The 
approach  can  be  easily  modified  by  adding  or  deleting 
complexity  elements  or  by  revising  the  time  estimates  as 
requirements  change.  The  approach  produces  indices  that 
are  directly  compatible  with  the  mission  of  the  orga¬ 
nization  and  its  outputs  can  be  defined  in  such  a  way 
that  the  total  mission  of  the  unit  can  be  captured. 

Finally,  in  its  implementation  it  is  almost  inevitable 
that  individuals  in  the  organization  become  involved  in 
development  of  the  procedure.  A  final  advantage  of  the 
procedure  is  that  there  should  be  high  agreement  among 
observers  as  to  whether  a  particular  complexity  element 
is  performed  which  would  lead  to  high  reliability. 

Use  of  historical  or  technical  audit  standards  has 
some  disadvantages.  First,  it  requires  a  considerable 
investment  of  time  to  develop  the  complexity  and  time 
estimates.  Second,  in  operation,  the  procedure  relies  on 
self-reports  of  incumbents.  Individuals  must  be  willing 
to  take  the  time  to  accurately  report  time  spent  and  work 
performed.  To  achieve  valid  reporting,  employees  must  be 
made  to  feel  that  it  is  in  their  best  interests  to  do  so. 

If  employees  feel  that  their  efforts  to  participate  in 
a  measurement  system  will  be  rewarded  by  getting  additional 
staff  or  in  other  ways,  the  quality  of  the  data  is  likely 
to  improve.  The  quality  of  the  output  data  can  be  assessed 
through  periodic  quality  control  checks  since  the -e  is  a 
clear  audit  trail. 


Utilization  Measures. 

1.  Description.  In  its  simplest  form,  a  measure  of  utilization 

is  the  ratio  of  actual  utilization  to  the  potential  utilization 
of  labor,  space,  equipment,  or  other  aspect  of  organizational 
capacity.  Some  utilization  ratios  drawn  from  physical  dis¬ 
tribution/logistics  include: 


Equipment  Utilization 


Equipment  hours  used  in  put-away 
Tot-"'!  equipment  hours  available 


Facility  Utilization 


Sg.  feet  of  storage  used _ 

So.  feet  of  storaae  available 
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Labor  Utilization  =  Labor  hours  spent  in  replenishing  stock 

Labor  hours  worked  by  replenishment  workers 

.  .  Total  volume  received  per  day 

Total  receiving  capacity  per  day 

2.  Critique .  In  general,  the  more  of  an  organization's  capacity 
being  utilized  the  better.  Certainly  this  is  true  for 
civilian  organizations.  However,  for  many  military  organi¬ 
zations,  particularly  those  which  have  wartime  missions, 
utilization  ratios  during  peacetime  may  not  be  meaningful. 

Units  which  are  authorized  equipment  for  wartime  use  would 
probably  not  find  high  rates  of  utilization  during  peacetime. 
Low  utilization  would  not  mean  that  the  units  were  in¬ 
efficient  but  rather  that  the  mission  workload  which  called 
for  the  utilization  of  the  equipment  was  not  present.  How¬ 
ever,  utilization  rates  for  personnel  and  equipment  required 
for  peacetime  missions  can  and  should  be  tracked. 

In  general,  utilization  rates  are  meaningful  indices  for 
managers  of  the  extent  to  which  valuable  assets  are  being 
utilized.  However,  it  is  important  that  consistent  definitions 
be  developed  in  order  to  have  accurate  recording  of  "capacity." 
For  example,  in  determining  equipment  availability,  is  the 
time  spent  conducting  preventive  maintenance  on  a  piece  of 
equipment  counted  as  time  when  the  equipment  is  utilized,  is 
it  counted  as  unavailable  time  to  be  subtracted  from  total 
time  available,  or  is  it  actually  counted  as  utilization 
time?  Clear  and  unambiguous  recording  procedures  must  be 
established  in  order  to  have  meaningful  utilization  measures. 
Such  measures  are  useful  in  assessing  the  effects  of  organi¬ 
zation  change  programs. 

Effectiveness  Measurement 

Effectiveness  has  been  defined  previously  as  the  extent  to  which  an 
organization  achieves  its  goals.  The  definition  can  be  broadened  to  in¬ 
clude  not  only  the  dimension  of  goal  achievement,  but  also  other  dimen¬ 
sions  such  as  quality  of  the  output,  impact  on  the  external  environment, 
and  impact  on  the  organization  itself.  Usilaner  and  Soniat  (1980)  list 
the  following  as  effectiveness  dimensions  relevant  to  the  services  of 
state  and  local  government:  responsiveness,  timeliness,  accessibility, 
availability,  participation,  safety,  reliability,  and  citizen  partici¬ 
pation  and/or  satisfaction.  Chapter  2  reviewed  the  concept  of  effective¬ 
ness  and  the  many  dimensions  that  researchers  have  proposed  and  measured. 
This  section  will  focus  principally  on  objective  measures  of  effective¬ 
ness  dimensions. 

Goal  or  mission  oriented  criteria  are  the  most  frequently  mentioned 
measures  of  effectiveness  by  managers.  This  was  illustrated  in  a  study 
of  68  Navy  Managers  (Bowser,  1976)  .  In  addition  to  the  criterion  of 
meeting  operational  commitments,  which  was  by  far  the  most  important 
dimension  of  concern,  Navy  managers  also  mentioned  efficiency,  training, 
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operational  readiness,  having  sufficient  resources,  operational  time 
factors,  safety,  communications,  adequate  maintenance, and  quality  of 
equipment  as  important  criteria. 

Interviews  with  23  Naval  Officers  attending  the  Nava!  pr,s< gra  luate 
School  yielded  the  following  list  of  measures  of  component  eff  •  •  .  'eness 
(Campbell,  et  al . ,  1974): 

Reenlistment  rate 

Non- judicial  punishment  rate 

Ratio  of  rated  to  non-rated  personnel 

Ratio  of  qualified  to  non-qualif ied  personnel 

Unauthorized  absence  rate 

First  tour  reenlistment  rate 

Long-term  stability  of  personnel 

Court  martial  rate 

Manning  level  ratio 

Percentage  taking  exams  who  pass 

Percentage  of  persons  eligible  for  promotion  who  are  recommended 
Rate  of  correspondence  course  participation 
Rate  of  participation  in  group  activities 

Number  of  times  per  person  per  ship  per  month  sick  call  rates 
Percent  time  sleeping  while  off  duty 
Rating  of  morale  by  top  officers 

Overall  rating  of  personnel  capability  by  top  officers 

Perceived  leadership  effectiveness  by  crew  members 

Food  and  personal  services  evaluation 

Reported  drug  usage 

Overall  ADMAT  inspection  grade 

Discharges  other  than  honorable 

Grievances  directed  to  others  outside  of  command  (letters  to 
Congress,  etc.) 

Percentage  of  maintenance  actions  deferred  due  to  insufficient 
manning  or  expertise 

Percentage  of  required  training  completed  by  officers 
Requests  for  transfer/per  man/per  unit  of  time. 

Satisfaction  with  present  assignment 
Satisfaction  with  shipmates 


r*  -  ■  i 
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As  this  list  suggests,  there  is  a  virtually  unlimited  number  of  possible 
criteria  of  effectiveness.  In  defining  and  measuring  effectiveness,  it 
is  helpful  to  have  some  structure  or  categories  of  effectiveness  measures. 
In  discussing  effectiveness  measurement.  Balk  (1975)  points  oat  that  the 
essence  of  effectiveness  is  a  comparison  between  an  output  and  some 
standard.  The  Urban  Institute  and  the  International  City  Management 
Association  (1974)  point  out  that  the  measurement  of  effectiveness  of 
municipal  services  includes: 

1.  the  degree  to  which  the  intended  purposes  of  the  service 

are  being  met 

2.  the  degree  to  which  unintended  adverse  impacts  of  the 

service  on  the  community  occur 

3.  the  adequacy  of  the  quantity  of  the  service  provided 

relative  to  the  community's  needs,  desires,  and  willingness 

to  pay 

4.  the  speed  and  courtesy  displayed  in  responding  to 

citizen  complaints 

5.  citizen  perceptions  of  the  satisfactoriness  of  the  service 
Goals  achieved. 

1.  Description.  The  first  category  of  effectiveness  measurement 
to  be  discussed  is  the  category  of  goal  achievement.  This 
measurement  approach  assumes  that  quantifiable  goals  have 
been  established  and  performance  is  assessed  relative  to 
those  goals  and  standards.  This  approach  is  in  common  use 
in  the  Air  Force.  For  example,  with  respect  to  aircraft 
availability: 

„  .  percentage  fully  mission  capable 

Aircraft  Availability  =  r - - — - - - - - - — — 

standard  percentage  for  Command 


This  measure  assesses  the  extent  to  which  the  number  of  air¬ 
craft  available  to  fly  missions  is,  on  the  average,  consistent 
with  the  percentage  expected  by  the  Major  Command.  With 
respect  to  aircraft  departure  reliability  from  the  origina¬ 
ting  station,  one  command  has  a  standard  of  98%.  Thus,  98% 
of  the  flights  originating  from  the  home  station  should  take 
off  on  time.  A  measure  of  effectiveness  for  this  goal  is: 


Departure  Effectiveness 


departure  reliability  percentage 
standard  percentage  for  Command 


In  addition  to  ratios,  goal  achievement  may  also  be  measured 
in  terms  of  adherence  to  schedules,  planned  achievements 
versus  actual  achievements,  etc. 
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2.  Critique.  Measuring  effectiveness  in  terms  of  performance 

against  goals  or  standards  is  a  straightforward,  understand¬ 
able  approach  as  long  as  an  organization  has  established 
measurable  goals  and  objectives  that  are  consistent  with 
the  mission  of  the  unit.  In  establishing  effectiveness 
measurement  procedures,  it  is  important  that  the  measurement 
operations  be  explicitly  defined.  For  example,  in  the 
first  illustration  (Aircraft  Availability),  whether  an  air¬ 
craft  is  fully  mission  capable,  or  partially  mission  capable 
can  involve  judgement.  To  the  extent  that  a  unit  is  under 
extreme  pressure  to  report  to  a  higher  headquarters  a 
number  that  is  in  compliance  with  the  Command  standard,  there 
would  be  a  tendency  to  make  the  percentage  look  as  good  as 
possible.  However,  having  explicity  defined  criteria  for 
mission  capable  aircraft  will  help  reduce  this  measurement 
bias.  In  addition,  when  measurements  are  used  as  the  basis 
for  rewards  and  punishments  to  organizations  rather  than  for 
diagnosis,  then  they  become  increasingly  susceptible  to  bias. 

Generally  measures  of  effectiveness  which  refer  to  the  achieve¬ 
ment  of  goals  are  measures  that  are  useful  to  managers  and 
are  understandable.  If  managers  participate  in  establishing 
goals,  and  if  they  are  considered  fair  and  attainable  by 
actions  under  the  control  of  the  organization,  then  these  goal 
oriented  measures  are  acceptable  to  managers.  The  cost 
effectiveness  of  goal  attainment  measures  is  high  if  organi¬ 
zations  have  delineated  and  prioritized  goals  and  are  measur¬ 
ing  those  things  which  are  really  important.  Frequently,  it 
is  better  to  measure  well  a  few  objectives  that  are  critical 
to  an  organization's  mission,  rather  than  try  to  measure 
everything  the  organization  does,  but  measure  it  poorly. 

From  the  point  of  view  of  evaluation,  measures  of  goal  attain¬ 
ment  often  have  less  utility  than  might  appear  because  there 
frequently  is  little  variation  in  the  measures.  For  example, 
in  a  squadron  flying  C-9  aircraft  over  a  12  month  period,  the 
departure  reliability  rate  fell  below  100%  (the  standard  is 
97%)  only  for  four  months.  In  those  months  the  rate  was 
99.4%,  97.9%,  99.0%,  and  99.5%.  As  a  research  criterion,  this 
would  not  be  very  useful  since  there  is  little  room  for  im¬ 
provement  in  this  measure.  This  is  frequently  the  case  in 
goal  areas  that  are  closely  monitored  by  higher  headquarters. 

Quality.  Definitions  of  quality  are  almost  as  elusive  as  definitions 
of  productivity.  However,  considerable  progress  has  been  made  recently 
in  defining  and  measuring  quality.  Adam,  Hershauer  and  Ruch  (1978)  have 
recently  defined  quality  as: 

...the  degree  to  which  a  product  or  service  conforms  to 
a  set  of  predetermined  standards  related  to  the 
characteristics  that  determine  its  value  in  the  market¬ 
place  and  its  performance  of  the  function  for  which  it 
was  designed  (p.  7)  . 
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While  this  definition  shifts  the  discussion  of  quality  from  a  consider¬ 
ation  of  quality  to  a  consideration  of  the  "standards",  it  does  help 
focus  the  issue. 

Following  the  approach  of  Adam,  et  al.  (1978),  quality  is  measured 
by  defining  key  quality  deviations  from  the  standards  of  acceptable 
performance.  In  a  later  paper,  Adam,  Herschauer  and  Ruch  (1981) ,  propose 
two  general  forms  of  quality  indices:  process  quality  and  prevention 
quality. 


1. 


Process  Quality  Ratios, 

a.  Description.  The  process  quality  productivity  ratio  has 
following  general  form: 


Process  Quality  = 
where 


pU  +  cE 

A  =  No.  of  units  acceptable  and  free  of  errors 
U  =  No.  of  units  processed 

E  =  No.  of  units  subjected  to  correction  procedures 
p  =  Processing  cost  per  unit 
c  =  Cost  per  unit  for  correction  processing 


b.  Critique .  This  is,  in  effect,  a  total  factor  productivity 
measure  that  considers  quality.  It  has  the  same  ad¬ 
vantages  as  the  total  factor  measure  discussed  previously 
In  addition,  this  measure  has  the  advantage  that  costs  of  re¬ 
processing  caused  by  errors  can  be  isolated  as  a  separate 
item.  In  organizations  which  have  error  detection  mechanisms, 
this  measure  can  be  applied  with  minimal  additional  cost. 

In  organizations  that  do  not  have  such  mechanisms,  additional 
cost  will  be  required  to  institute  error  detection  systems. 
Many  Air  Force  organizations  have  quality  control  procedures 
and  could  make  use  of  this  measure.  Examples  are  maintenance 
organizations,  administrative  organizations  which  process 
"cases"  such  as  personnel,  finance,  etc.  Organizations 
whose  output  is  an  input  to  an  automated  system  can  make 
use  of  computer-generated  error  reports.  A  process  quality 
measure  is  useful  to  managers  as  a  source  of  feedback  and 
it  is  a  very  useful  criterion  for  evaluating  organizational 
performance.  One  frequently  occurring  outcome  of  organiza¬ 
tional  change  is  an  improvement  in  product  or  service  quality. 

2 .  Prevention  Quality  Ratio. 

a.  Description.  The  prevention  quality  productivity  ratio  is 
an  index  of  the  cost  per  unit  of  prevention  service.  The 
general  form  of  this  index  is: 

S 

Prevention  Quality  =  -  -  —  -+  —  • 
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where:  S  =  Percent  of  "units"  satisfactorily  serviced  during  the  period 

D  =  Development  cost  allocated  for  this  prevention  system 

M  =  Maintenance  cost  per  period  to  execute  the  prevention  system 

Z  =  Special  costs  required  by  the  prevention  system 

This  general  form  can  be  used  to  assess  the  productivity 
of  any  error  prevention  subsystem.  Examples  of  such 
subsystems  include  quality  control,  consumer  protection, 
safety,  and  affirmative  action. 


b.  Critique .  This  index  represents  one  way  to  assess  the 

effectiveness  of  efforts  to  prevent  errors.  In  effect  it 
provides  a  means  for  tracking  the  cost  of  preventing 
problems.  The  unit  chosen  can  be  people  serviced,  pro¬ 
ducts  inspected,  community  members  protected,  etc.  Making 
such  a  measure  operational  will  usually  require  addition¬ 
al  effort  to  gather  cost  data  in  the  denominator  and  to 
define  the  population  satisfactorily  serviced.  Develop¬ 
ing  an  adequate  measure  of  thenumber  of  units  successfully 
serviced  is  a  difficult  problem.  For  example,  does  the 
absence  of  a  complaint  mean  that  a  person  was  serviced 
adequately?  Does  the  absence  of  an  accident  mean  that 
the  safety  program  is  effective?  How  does  one  assess  the 
number  of  accidents  that  might  have  occurred  with  no  invest 
ment  in  a  safety  program?  These  are  some  of  the  questions 
which  must  be  answered  in  operationalizing  this  index.  De¬ 
spite  these  concerns,  the  index  can  provide  valuable  cost 
visibility  for  programs  of  prevention. 


Schaenman  and  Swartz  (1974)  suggest  the  use  of  a  variation 
of  the  prevention  quality  index  in  evaluating  the  effective 
ness  of  fire  prevention  services: 


Fire  incidence  rate  per  1,000  population 

Fire  Prevention  Effectiveness  =  cm - -  , . ■  - ; — • — 

Fire  prevention  expenditures  per  1,000 

population 


These  authors  suggest  that  this  index  be  viewed  over  time 
in  order  to  make  it  meaningful.  The  fire  prevention 
index  focuses  on  the  number  of  "errors"  as  opposed  to  the 
number  "properly  serviced."  Since  organizations  are  more 
likely  to  keep  data  on  mistakes,  complaints,  rejects, 
accidents,  etc.  than  on  the  "number  of  buildings  that  do 
not  burn"  or  the  "number  of  training  flights  without  an 
accident,"  it  would  seem  that  the  error  measurement 
approach  would  be  less  costly  and  more  acceptable  to 
most  organizations.  Unfortunately,  interpretation  of  the 
index  is  not  as  straightforward  as  the  prevention  quality 
index.  With  respect  to  their  relevance  for  evaluating 
organizational  change  programs,  neither  the  prevention 
quality  nor  the  prevention  effectiveness  index  is  likely 


to  be  very  useful  except  in  organizations  where  "errors" 
are  rather  frequent  events.  Most  criteria  of  this  sort 
(e.g.,  accidents,  fires,  injuries)  are,  fortunately,  rare 
events;  therefore,  the  measures  have  relatively  little 
variance.  As  a  result,  this  form  of  quality  index  would 
not  likely  be  sensitive  to  improvements  in  organizational 
effectiveness  resulting  from  change  programs. 

Conformance  to  Quality  Standards. 

a.  Description.  The  most  common  way  of  assessing  quality  is 
in  relation  to  established  standards.  The  mechanism  for 
determining  conformity  in  the  Air  Force  is  typically 
through  inspections,  although  other  measurement  approaches 
are  also  used.  For  example,  response  time  to  a  call  may 
be  a  quality  indicator  for  the  base  taxi  service.  If 

the  standard  response  time  is  4  minutes  with  an  allowable 
deviation  of  5%,  it  is  possible  to  determine  through 
dispatcher  records  whether  the  average  response  time  lies 
within  the  acceptable  range. 

In  another  area  such  as  base  housing,  the  maintenance  may 
be  performed  either  by  a  contractor  or  by  government 
personnel.  A  quality  indicator  for  the  required  "change 
of  occupancy  maintenance"  (that  maintenance  required  to 
prepare  a  residence  unit  for  the  next  occupant)  is  the 
percentage  of  units  turned  over  that  require  rework.  With 
a  standard  set  at  4%  rework  as  acceptable,  conformance  to 
the  standard  is  monitored  by  100%  inspection  of  units  by 
the  Family  Housing  Management  Office. 

In  the  area  of  procurement,  processing  time  is  a  quality 
index.  Procurement  managers  monitor  the  average  cycle 
time  for  various  types  of  procurement  actions.  In  one 
MAJCOM,  data  is  regularly  maintained  for  the  number  of 
days  required  to  process  small  purchases  under  $10,000. 

The  MAJCOM  has  set  a  standard  of  50  days,  and  monitors  the 
the  average  number  of  days  required  on  a  monthly  basis. 

This  average  is  reported  and  graphed  monthly  in  relation 
to  the  standard. 

b.  Critique.  Conformance  to  standards  of  quality  is  the 
essence  of  quality  measurement.  The  advantages  of  this 
approach  are  that  it  forces  an  organization  to  make  re¬ 
levant  quality  dimensions  and  standards  explicit.  When 
applied  to  important  organizational  outputs,  quality 
measures  are  valuable,  have  high  management  acceptance, 
and  are  potentially  useful  indicators  for  evaluation. 
However,  if  they  are  to  have  these  positive  attributes, 
the  indicators  must  be  realistic,  and  the  dimension  must 

be  controllable  through  actions  taken  by  the  organization's 
members.  For  example,  in  aircraft  maintenance,  maintenance 
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induced  delays  in  departures  are  a  quality  indicator. 
However,  if  maintenance  fails  to  complete  a  repair 
operation  because  it  has  to  wait  for  supply  to  get  a 
needed  part  to  the  flight  line,  then  this  quality 
measure  will  not  be  viewed  as  "fair"  by  maintenance 
personnel.  Nevertheless,  conformance  to  quality  stan¬ 
dards  can  be  a  very  useful  management  and  evaluation 
criterion.  With  respect  to  evaluation,  however,  such 
criteria  sometimes  suffer  from  the  problem  mentioned 
previously  of  having  insufficient  variation  to  be  sen¬ 
sitive  to  changes.  This  is  particularly  the  case  in  areas 
where  the  consequences  of  an  error  are  very  serious. 


Impact  — External,  In  addition  to  whether  an  organization  achieves 
its  goals  and  produces  products  and  services  which  conform  to  specifi¬ 
cations,  one  must  consider  the  impact  which  an  organization's  outputs 
have  on  its  environment.  In  some  cases,  impact  can  be  directly  assessed; 
in  other  cases,  it  is  difficult  or  impossible  to  assess.  For  example, 
the  impact  of  a  flight  line  refueling  operation  on  a  wing's  aircraft 
departure  reliability  can  be  directly  assessed  by  counting  the  number  of 
times  flights  are  delayed  due  to  flight  line  refueling.  However,  the 
impact  of  a  professional  military  school’s  educational  programs  on  its 
graduate's  performance  in  future  assignments  is  much  more  difficult  to 
assess.  This  section  will  consider  three  approaches  to  assessing  impact 
of  an  organization's  output:  direct  or  indirect  impact  on  another 
organizations' s  performance,  responsiveness  to  customers/users,  and 
impact  on  customer  attitudes. 


Assessing  Impact  On  Another  Organization's  Performance 


a.  Description.  One  type  of  impact  measure  is  the  extent  to 
which  the  output  of  an  organization  has  an  effect  on  the 
performance  of  another  organization  which  depends  on  the 
first  organization's  product  or  service.  To  illustrate 
this  type  of  measure,  consider  the  impact  of  an  avionics 
maintenance  squadron  on  the  departure  reliability  index 
for  a  wing.  Delayed  departures  due  to  avionics  components 
are  an  example  of  the  squadron’s  impact  on  the  performance 
of  the  wing  as  measured  by  departure  reliability.  For 
example : 


Impact  = 


Number  of  takeoffs  delayed  or  aborted  due  to  avionics 
Number  of  avionics  maintenance  actions  completed 


Another  example  would  be  the  number  of  times  organization 
A ' s  output  is  rejected  by  organization  B.  For  example, 
an  aircraft  maintenance  shop  may  subcontract  fabrication 
work  to  a  machine  shop.  The  machine  shop's  impact  could 
be  assessed  in  terms  of  the  percentage  of  components 
produced  which  were  rejected  by  the  receiving  maintenance 
organization's  quality  inspection. 
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b.  Critique .  Measurement  of  the  impact  of  an  organizaion ' s 
outputs  on  the  performance  of  the  consuming  organizations 
is  an  "ultimate  criterion  of  effectiveness."  When  the 
impact  is  direct,  such  as  in  the  maintenance  rejects 
example  above,  this  is  a  very  useful  measure  to  managers. 
When  the  assessment  is  indirect,  its  usefulness  and 
acceptability  are  dependent  on  the  plausibility  of  the 
cause-effect  relationship  between  the  organization's 
performance  and  the  measure.  When  this  is  contaminated 
by  other  influences,  many  managers  find  such  indicies  to 
be  necessary  but  not  as  helpful  to  them  in  guiding 
management  practice  as  other  more  direct  measures  of 
efficiency  and  effectiveness.  In  essence,  some  managers 
might  say  that  "if  you  do  the  job  right,  impact  will  take 
care  of  itself." 

The  same  factors  that  cause  managers  to  play  down  the 
importance  and  utility  of  indirect  measures  of  impact 
also  restrict  their  utility  as  evaluation  criteria.  For 
example,  in  the  previous  avionics  example,  the  avionics 
squadron  could  perform  its  job  accurately  and  efficiently 
but  still  be  "charged"  with  a  delayed  departure  due  to 
factors  beyond  its  control,  such  as  the  performance  of  some 
other  unit  (e.g.,  supply  not  being  able  to  get  a  part  on 
time) .  Such  criterion  contamination  reduces  the  utility 
of  impact  criteria  for  evaluation  purposes. 


a.  Description  .  Responsiveness  can  be  assessed  in  a  variety 
of  ways.  Many  organizations  use  objective  indicators  of 
responsiveness,  timeliness  of  response,  and  measures  of 
customer  behavior.  A  private  air  freight  firm  records 
and  monitors  indices  of  responsiveness  such  as  (Bryant, 
Brewer,  Beasley,  Wagner,  and  Adlfinger,  1972) : 

Percentage  of  incoming  calls  answered  in  30  seconds 

Percentage  of  customers  called  back  in  90  minutes 

A  data  processing  work  center  monitors  indicators  such 
as  (Kirkbride,  1977): 

Percentage  of  bills  paid  per  month  without  user  complaint 
Average  time  job  received  versus  desired  completion  time 
Number  refunds  requested 
Percentage  of  repeat  complaints 

b.  Critique .  Responsiveness  indicators,  when  measured  over 
time,  have  the  advantages  of  objectivity  and  sensitivity 
to  performance  change.  Measures  such  as  "Percent  of 
calls  answered  in  30  seconds"  are  preferable  to  those 
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depending  on  customer  behavior  because  they  measure  per¬ 
formance  that  is  controllable  by  the  organization.  Re¬ 
sponsiveness  measures  that  are  linked  closely  to  the 
behavior  of  organizational  members  are  extremely  useful 
to  managers,  are  acceptable,  and  make  good  evaluation 
criteria.  The  only  disadvantage  is  that  some  effort  and 
cost  is  required  to  develop  behavior  recording  procedures. 

In  most  cases,  the  advantages  of  measuring  such  key  re¬ 
sponsiveness  behaviors  outweigh  the  costs  of  data 
collection.  Customer  behavior  measures  are  difficult  to 
interpret  unless  the  organization  maintains  longitudinal 
data  which  allows  trends  to  be  detected. 

Impact  —  Internal.  The  criteria  discussed  to  this  point  have 
focused  on  goal-centered  measures  of  efficiency  and  effectiveness.  It 
is  also  important  to  consider  process  criteria — the  extent  to  which  the 
organization's  capability  for  future  production  is  enhanced  or  impaired. 

There  are  no  direct  measures  of  this  future  productive  capacity.  However, 
the  "health"  of  the  organization  can  be  inferred  from  assessment  of 
indices  of  absenteeism,  requests  for  transfer,  non- judicial  punishment 
rates,  accidents,  stress  related  health  problems,  etc.  Other  internal 
impact  measures  include  perceptions  and  attitudes  of  organizational 
members  on  topics  such  as  reactions  to  job  characteristics,  working 
conditions,  adequacy  of  supervision,  interpersonal  relationships, 
participation  in  decision  making,  adequacy  of  planning,  goal  setting  and 
performance  feedback  and  recognition  for  accomplishments.  At  the  same 
time  organizational  results  are  monitored,  it  is  important  to  measure 
these  process  indicators  to  ensure  that  the  organization's  "productive 
capacity"  is  being  maintained.  In  the  Air  Force  environment,  useful 
measures  of  such  process  criteria  have  been  previously  developed 
(Hendrix  &  Halverson,  1979;  Gould,  1978;  Tuttle,  Gould,  &  Hazel,  1975)  . 

Integrated  Approaches 

The  previous  section  emphasized  the  need  to  measure  not  only  goal- 
centered  criteria,  but  also  process  criteria.  The  goal-centered  criteria 
provide  indicators  of  what  the  organization  has  accomplished  in  the  past. 
Process  criteria  provide  information  on  the  "health  of  the  organization." 

The  organization's  health  is  an  indicator  of  what  the  organization  can 
be  expected  to  accomplish  in  the  future  (Likert,  1977) . 

Measurement  approaches  which  recognize  this  distinction  and  the 
importance  of  integrating  measures  of  both  types  are  Total  Performance 
Management  (National  Center  for  Productivity  and  Quality  of  Working  Life, 
(NCPQWL)  1978),  Behavioral-Economic  Measurement  (Herrick,  1975;  Macy  and 
Mirvis,  1976),  and  a  Navy  system  for  evaluating  the  performance  of  production 
foremen.  These  three  approaches  will  be  discussed  in  this  section. 

Total  Performance  Management  (TPM) . 

1.  Description.  Formerly  called  Total  Performance  Measurement, 

this  process  integrates  procedures  from  industrial  engineering 
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and  behavioral  sciences  into  a  comprehensive  management 
process.  TPM  has  five  basic  components  (NCPQWL,  1978): 

1 .  The  employee  survey 

2.  The  customer  or  user  survey 

3.  Measurement  of  efficiency,  effectiveness 

4.  Information  feedback  to  managers  and  employees 

5.  Planning  of  action  to  correct  deficiences 

TPM  differs  from  traditional  performance  measurement  in  at 
least  three  ways.  First,  it  is  based  on  the  assumption  that 
workers  are  the  best  source  of  information  of  their  performance 
and  work  situation  and  on  how  these  can  be  improved.  Secondly, 
performance  data  are  collected  at  regular  intervals  in  order 
to  spot  emerging  problems  before  they  become  crises.  Third 
TPM  provides  a  means  for  viewing  specific  problems  in  relation 
to  the  total  organization.  Fourth,  TPM  data  is  made  available 
to  everyone  in  the  organization. 

Proponents  of  TPM  suggest  that  there  are  four  questions  that 
should  be  asked  in  order  to  determine  an  organization's  readi¬ 
ness  for  TPM  (NCPQWL,  1978) : 

1.  Are  valid  productivity  data  routinely  available?  If  not 

this  is  the  place  to  start.  Trying  to  implement  TPM 
at  the  same  time  basic  productivity  measurement  is  in¬ 
troduced  may  be  too  much  for  the  organization  at  one  time. 

2.  What  resources  are  available  to  implement  TPM?  Experience 

with  the  process  suggests  that  there  must  be  a  full-time 
coordinator  who  has  assistance  from  individuals  with  ex¬ 
pertise  in  industrial  engineering  and  behavioral  science. 
Money  must  be  allocated  for  the  coordinator,  consultants, 
or  advisors,  and  for  other  direct  costs  such  as  printing 
surveys,  clerical,  and  computer  assistance  in  administer¬ 
ing  surveys  and  analyzing  results. 

3.  Are  there  time  limits  on  the  project?  Managers  looking 

for  a  "quick  fix"  of  their  productivity  problems  should 
not  use  TPM.  Usually,  5  to  6  months  are  required  to 
plan,  collect,  and  analyze  survey  data.  The  full  process, 
including  feedback  and  development  of  solutions,  takes 
longer.  Including  assessment  of  results,  2  years  is  a 
realistic  time  frame  for  a  TPM  project. 

4.  Will  the  organization  support  the  process?  Top  management 

must  support  the  process  from  start  to  finish.  The 
process  can  be  threatening  in  that  it  will  shift 
communication  patterns  from  "top-down"  to  "bottom-up" 
approaches.  Managers  must  expect  to  hear  criticism  and 
must  be  prepared  to  deal  with  it  constructively.  A 
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frequent  comment  from  organizations  involved  in  TPM  was 
stated  by  one  site  coordinator.  (NCPQWL,  1978,  p.  5) 

As  more  people  get  more  involved,  we  really 

picked  up  momentumm . but  we  really  didn't 

know  what  we  were  getting  into  when  we  began. 


The  remainder  of  this  discussion  will  focus  on  the  measurement 
approach  which  underlies  TPM.  The  measurement  philosophy  is 
summarized  in  the  following  paragraph  from  the  TPM  manual 
(NCPQWL,  1978,  p.2)  which  says  that: 

Total  Performance  Management  emerged  from  the  re¬ 
alization  that  both  approaches,  industrial  engineer¬ 
ing  and  behavioral  science,  provide  useful  informa¬ 
tion  when  combined  in  a  systematic  way.  By  using 
productivity  measures  and  data  on  employee  attitudes 
in  combination  as  well  as  data  about  customer  satis¬ 
faction,  managers  can  obtain  a  more  realistic 
picture  of  their  organization' s  performance  problems 
and  potentials.  By  providing  this  information  to 
employees,  and  seeking  their  recommendations,  man¬ 
agers  can  open  avenues  to  plan  for  change. 

a.  Measures  of  efficiency.  The  primary  measures  of  efficiency 
in  TPM  have  been  measures  of  output  per  hour  of  labor 
input.  Efficiency  trends  are  normally  assessed  by  con¬ 
verting  efficiency  ratios  to  index  form  for  comparison 
across  time  periods.  Table  4  illustrates  hypothetical 
eft  icienvy  data  from  a  TPM  protect  . 

In  computing  efficiency  measures,  use  of  total  resource 
input  in  the  denominator  rather  than  partial  resource  in¬ 
puts  is  recommended.  Using  total  resources  can  give  a 
more  accurate  view  of  efficiency  than  reliance  on  partial 
input  data.  Table  5  illustrates  this  point.  Results 
from  the  administration  department  demonstrate  the 
optimal  result.  Output  per  hour  worked  has  gone  up  while 
total  cost  per  unit  of  output  (the  total  resource  measure) 
has  declined.  The  library,  on  the  other  hand,  has  in¬ 
creased  labor  efficiency,  but  the  cost  per  unit  of  output 
has  gone  up  at  an  even  faster  rate.  This  is  an  undesir¬ 
able  situation  that  would  not  have  been  obvious  from 
measures  of  labor  efficiency  alone. 
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Table  4 


Labor  Efficiency  Indices  for  Three  Years3 


Base  Year 

Year  2 

Year  3 

Weighted  Output 

3600 

4400 

4650 

Output  Index 

100 

122 

129 

Input 

4200 

4200 

4325 

Input  Index 

100 

100 

102 

Efficiency  Rate 

.857 

1.047 

1.075 

Efficiency  Index 

100 

122 

126 

3  Adapted  from  National  Center  for  Productivity  and  Quality 

of  Working 

Life,  1978,  p.15. 

Table 

5 

Comparison  of 

Labor  Efficiency  and 

Total  Cost  Per  Unit 

of  Output3 

Annual  Percent  Change,  1974-1976 

Department 

Labor  Efficiency 

Total  Cost  Per  Unit  of  Output 

Administration 

+  7.4 

-  6.9 

Finance 

+  6.2 

+  1.0 

Library 

+  4.8 

+  6.1 

Recreation 

-  0.1 

-  2.3 

Public  Works 

+  2.9 

+  1.1 

aAdapted  from  National  Center  for  Productivity  and  Quality  of  Wonting 
Life,  1978,  p.  20. 


Measures  of  effectiveness.  Effectiveness  measures  take 
into  account  the  extent  to  which  an  organization  fulfills 
its  mission,  the  extent  to  which  it  delivers  high  quality 
products  and  services,  and  the  organization's  impact  on 
other  organizations,  customers,  and  its  own  employees. 

The  previous  discussion  of  effectiveness  measures  provides 
some  illustrations  of  their  use  and  some  considerations 
in  developing  valid  measures.  In  TPM,  the  focus  is  on 
objective  measures  of  effectiveness,  e.g.,  goals  accom¬ 
plished,  response  time,  percent  rejects,  scrap  produced. 

Employee  survey .  The  employee  survey  obtains  information 
from  workers  and  managers  regarding  the  functioning  of 
the  organization  and  their  reactions  to  working  in  the 
organization.  There  are  many  previously  developed 
surveys  that  address  such  issues.  Two  which  have  been 
developed  specifically  for  Air  Force  organizations  are  the 
Occupational  Atittude  Inventory  (Gould,  1978;  Tuttle, 

Gould,  &  Hazel,  1975)  and  the  Organizational  Assessment 
Package  (Hendrix  &  Halverson, 1979) . 

Customer  survey.  In  TPM  the  rationale  underlying  the 
customer  survey  is  that  users  of  products  or  services 
are  in  the  best  position  to  judge  their  value. 

Questions  in  the  survey  should  address  aspects  of  the 
organization's  effectiveness  that  are  important  to 
consumers  and  are  under  the  organization’s  control, 
such  as  timeliness,  accuracy,  completeness,  and  courtesy. 
The  survey  also  gathers  background  information  on  the 
respondent  to  assess  the  individual's  knowledge  of  the 
organization,  degree  of  contact  with  products  and  services, 
and  other  demographic  data  that  would  help  identify  the  re¬ 
spondent's  point  of  view  (e.g.,  income  status  and  education 
level)  . 

Combining  data  sources.  Once  the  data  are  gathered,  there 
must  be  a  plan  of  analysis  in  order  to  make  sense  out 
of  the  data.  One  way  to  organize  the  information  is  in 
terms  of  "End  Results"  and  "Targets  of  Opportunity."  End 
results  focus  on  the  accomplishments  of  the  organization 
and  targets  of  opportunity  represent  areas  of  needed 
improvement . 

Comparisons  provide  a  useful  way  to  analyze  and  organize 
the  TPM  data.  Specifically,  it  may  be  helpful  to  compare 
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1.  results  for  individual  units  with  overall  organization 

averages 

2.  employee  attitudes  with  customer  attitudes 

3.  employee  attitudes  in  efficient  and  effective  units 

with  attitudes  of  employees  in  less  efficient  and 

effective  units 

4.  managers  and  hourly  workers  (or  officer  and  enlisted) 

perceptions 

5.  all  groups  over  time 

2.  Critique  .  TPM  provides  a  means  for  integrating  performance 
measurement  data  into  an  overall  productivity  improvement 
effort.  Using  any  one  of  the  types  of  measurement  alone 
runs  the  risk  of  providing  an  incomplete  picture  of  the 
organization's  performance.  This  approach  provides  data 
that  are  highly  useful  for  managers  in  guiding  their  manage¬ 
ment  practices  to  improve  productivity.  Since  organization 
effectiveness  can  be  viewed  from  at  least  three  perspectives, 
management,  employees,  and  the  society  at  large,  TPM  provides 
a  means  of  reflecting  these  multiple  viewpoints.  Having 
multiple  data  sources  tends  to  improve  the  validity  of  the 
data  for  diagnostic  purposes.  Another  major  advantage  is  that 
the  process  encourages  employee  input  to  the  measurement 
process  which  will  not  only  lead  to  better  measures,  but  will 
also  make  the  results  more  acceptable  to  all  members  of  the 
organization.  This  is  a  prerequisite  for  meaningful  change. 

For  TPM  to  be  successful  within  an  organization,  the  organi¬ 
zation  must  be  ready  to  commit  the  resources  and  to  open 
itself  up  to  in-depth  scrutiny.  The  process  is  likely  to 
stimulate  resistance  from  managers  who  are  uncomfortable 
with  criticism.  The  process  could  require  a  considerable 
investment  of  resources.  For  example,  the  costs  incurred  by 
one  TPM  project  implemented  in  a  city  totaled  almost  $80,000 
(NCPQWL,  1978).  Over  half  of  these  costs  ($47,000)  were  in- 
kind  costs.  Furthermore,  costs  assume  the  existence  of 
productivity  data,  and  include  not  only  measurement  but  also 
feedback  and  improvement  planning. 

In  many  ways,  TPM  provides  a  "blueprint"  for  evaluating  the 
impact  of  an  organizational  change  effort.  A  comprehensive 
evaluation  design  should  include  the  types  of  measurement 
data  suggested  by  this  procedure.  When  carefully  developed 
and  collected  over  time,  the  multiple  measures  can  provide 
a  valid  and  meaningful  set  of  evaluation  criteria. 

Behavioral  Economic  Measurement 

1.  Description  While  TPM  represents  a  blending  of  behavioral 

science  procedures  and  industrial  engineering  approaches,  the 
behavioral  economic  measurement  process  is,  as  the  name  implies. 
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a  blend  of  behavioral  and  cost  accounting  procedures.  The 
approach  involves  developing  a  set  of  behavioral  measures  that 
can  be  converted  to  dollar  values.  Since  Brogden  and  Taylor 
(1950)  first  proposed  the  "dollar  criterion,"  behavioral 
scientists  have  been  slow  to  adopt  this  approach.  A  more 
recent  attempt  which  stimulated  this  behavioral-economic 
approach  to  evaluating  quality  of  working  life  programs  was 
described  by  Herrick  (1975) .  Herrick  identified  ten  aspects 
of  counter  productive  labor  activity  that  could  be  measured. 
These  counter  productive  activites  (CPAs)  were  as  follows: 

(a)  absenteeism, 

(b)  accidents, 

(c)  tardiness, 

(d)  turnover, 

(e)  strike  days  lost, 

(f)  grievances, 

(g)  inventory  shrinkage, 

(h)  machine  repair, 

(i)  quality  below  standard,  and 

(j)  production  below  standard. 

In  addition,  Herrick  proposed  computational  formulas  and 
provided  detailed  examples  of  procedures  to  convert  these 
CPA  measures  to  dollar  values. 

Following  Herrick's  lead,  Macy  and  Mirvis  refined  this  list 
and  developed  a  set  of  operational  criteria  that  were  applied 
to  the  evaluation  of  a  quality  of  working  life  program 
(  Macy  and  Mirvis,  in  press;  Mirvis  and  Macy,  1976).  These 
authors  defined  a  standard  set  of  behaviors  that  conform  to 
three  criteria: 

(a)  definable  so  that  incidents  that  could  be  significantly 
affected  by  the  work  environment  are  distinguished  from 
those  unaffected. 

(b)  measurable  and  convertible  to  significant  costs  to  the 
organ  izat ion . 

(c )  categories  of  measures  and  costs  of  the  behaviors  are 
mutuaLly  exclusive. 

TabLe  6  lists  the  behavioral  definitions  and  the  computational 
formulas  for  these  indices.  Each  of  the  indices  can  be 
converted  to  dollar  values  by  calculating  from  the  organiza¬ 
tion's  cost  accounting  system  the  cost  per  incident  and 
multiplying  this  cost  by  the  total  number  of  incidents. 
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Table  6 


Behavioral  Definitions  and  Computational  Formulas 


Behavioral  Definitions 


Absenteeism: 

Each  absence  or  illness  over 
4  hours. 


Tardiness : 

Each  absence  or  illness  under 
4  hours 


Turnover: 

Each  movement  beyond  organiz¬ 
ational  boundaries. 

Internal  Employment  Stability: 

Each  movement  within 
organizational  boundary. 

Strikes  and  Work  Stoppages: 

Each  day  lost  due  to  strike  or 
work  stoppage . 


Accidents  and  Work  Related  Illness : 

Each  recordable  injury,  illness 
or  death  from  a  work  related 
accident  or  from  exposure  to 
the  work  environment. 

Grievance : 

Written  grievance  in  accordance 
with  labor-management  contract. 


Computational  Formulas 


Absenteeism  Rate: 

Number  of  Absence  Days _ 

Average  Work-Force  Size  x  Working 
Days 

Tardiness  Rate : 

Number  of  Tardiness  Incidents _ 

Average  Work-Force  Size  x  Working 
Days 

Turnover  Rate : 

Number  of  Turnover  Incidents 
Average  Work-Force  Size 

Internal  Stability  Rate: 

Number  of  Internal  Movement  Incidents 
Average  Work-Force  Size 

Strike  Rate: 

Number  of  Striking  Workers  x 

Number  of  Striking  Days _ 

Average  Work -Force  Size  x 
Working  Days 

Accident  Rate : 

Number  of  Accidents  , 
and  Illinesses  x  200,000° 

Total  Yearly  Hours  Worked 


Grievance  Rate : 

Plant:  Number  of  Grievance  Incidents 
Average  Work-Force  Size 

Individual:  Number  of  Aggrieved 
Individuals 

Average  Work-Force  Size 
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Table  6  (Con't) 


Behavioral  Definitions  and  Computational  Formulas 


a 


Behavioral  Definitions 


Computational  Formulas 


a 


Productivity;' 

Resources  used  in  production  of 
acceptable  outputs  (comparison 
of  inputs  with  outputs) . 


Production  Quality: 

Resources  used  in  production 
of  unacceptable  output. 


Downtime : 

Unscheduled  breakdown  of 
machinery. 

Inventory; 


Productivity: 

Total : 

Output  of  Goods  or  Services  "(Units 

_ or  $) _ 

Direct  and/or  Indirect  Labor  (Unit 
or  $) 


Below  Standard: 

Actual  versus  Engineered  Standard 
Below  Budget; 

Actual  versus  Budgeted  Standard 
Variance: 

Actual  versus  Budgeted  Variance 
Per  Employee: 

Output /Average  Work-Force  Size 


Quality: 
Total : 


Scrap  +  Customer  Returns  +  Rework 
Recoveries  ($,  Units,  or  Hours) 


Below  Standard; 

Actual  versus  Engineered  Standard 
Below  Budget: 

Actual  versus  Budgeted  Standard 


Variance: 

Actual  versus  Budgeted  Variance 
Per  Employee: 

Total/Average  Work-Force  Size 


Downtime : 

Labor  ($)  +  Repair  Costs  or  Dollar 
Value  of  Replaced  Equipment  ($) 

Inventory,  Supply,  and  Material 
Usage: 


Material  and  supply  variance; 
Unscheduled  resource  utilization 


Variance  (Actual  versus  Standard 
Utilization  [$]) 


2.  Critique.  Advantages  of  the  behavioral  economic  approach  are 
that  it  provides  a  standard  set  of  criteria  that  are  carefully 
defined,  applicable  across  organizational  settings,  meaningful, 
and  acceptable  to  managers.  The  approach  yields  measures 
that  are  expressed  both  in  physical  quantity  terms  and  in 
dollar  terms.  For  evaluation  purposes,  physical  quantity 
measures  are  likely  to  be  more  useful.  However,  if  organiza¬ 
tional  change  occurs,  dollar  terms  will  help  communicate  the 
significance  of  the  change  to  managers.  Thus,  the  behavioral 
economic  approach  represents  a  major  advance  in  techniques  for 
assessing  the  impact  of  organizational  change  efforts. 

The  behavioral  categories  defined  by  Macy  and  Mirvis  include 
indices  that  could  be  classified  as  both  efficiency  and 
effectiveness  according  to  the  definitions  used  in  this  review. 

Efficiency  Indices 

productivity 

inventory,  supply,  and  material  utilization 

Effectiveness  Indices 
absenteeism 
tardiness 
turnover 

internal  stability  rate 

strike  rate 

accident  rate 

grievance  rate 

quality 

downtime 

From  an  evaluation  point  of  view,  the  criteria  have  content 
validity  in  that  they  cover  major  behaviorally  related 
dimensions  of  organizational  performance. 

The  approach  has  some  disadvantages.  It  does  not  include 
assessment  of  performance  in  relation  to  overall  organizational 
goals  in  areas  covered  by  the  measures.  This  criticism  is  a 
bit  unfair,  in  that  the  authors  attempted  to  define  general 
categories  that  were  applicable  to  a  wide  range  of  organiza¬ 
tions.  For  research  purposes  this  is  necessary.  However, 
from  the  point  of  view  of  a  manager  who  wishes  to  measure  the 
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performance  of  his  or  her  organization,  additional  measures 
would  be  required.  One  example  of  an  organizational  specific 
criterion  that  is  missing  is  an  assessment  of  the  impact  of  the 
organization  on  its  external  environment.  Except  for  the 
turnover  index,  these  indices  look  at  the  organization  as 
separate  from  its  environment. 

There  are  also  some  disadvantages  associated  with  converting 
behavioral  indices  to  dollar  values.  Many  of  these  have  been 
discussed  previously.  The  two  main  problems  are  correcting 
for  the  effects  of  inflation  and  establishing  valid  and 
meaningful  costs  per  incident.  This  later  problem  is  less  a 
concern  for  productivity  (efficiency)  measures  than  for 
measures  such  as  grievances,  absenteeism  and  tardiness. 

Costing  the  effects  of  such  behavioral  variables  requires  use 
of  human  resource  accounting  principles.  New  recordkeeping 
systems  may  have  to  be  developed  to  provide  necessary  cost 
data  for  such  computations.  Macy  and  Mirvis  (in  press) 
acknowledge  these  problems  and  provide  a  very  detailed  analysis 
of  the  procedures  required  to  generate  meaningful  data. 
Nevertheless,  establishment  of  the  required  accounting  systems 
will  entail  additional  costs  to  organizations.  However,  the 
costs  should  not  be  exorbitant  or  out  of  line  with  the  benefits 
to  be  obtained. 

While  very  appropriate  for  private  sector  organizations,  some 
of  the  proposed  criteria  are  not  applicable  to  the  Air  Force 
environment,  e.g.,  strikes  and  grievances,  except  in  mostly 
civilian  unionized  settings.  Other  categories  would  have  to 
be  redefined  for  use  in  the  military  environment,  for  example, 
turnover  could  be  redefined  as  reenlistment  rate.  Neverthe¬ 
less,  the  philosophy  underlying  the  behavioral  economic  ap¬ 
proach  is  very  applicable  to  military  organizations.  Ap¬ 
propriately  modified,  such  an  approach  would  be  very  useful 
for  the  evaluation  of  organizational  change  programs  in 
Air  Force  organizations. 

Production  Foreman  Performance  Appraisal. 

1.  Description.  In  one  of  the  Navy's  major  maintenance  facilities 
employing  civilian  workers,  a  system  was  developed  for  ap¬ 
praising  the  performance  of  shop  foremen.  In  effect,  it  is 
a  system  for  measuring  efficiency  and  effectiveness  at  the 
work  group  level . 

The  system  design  effort  had  three  goals  (Morgan,  1979) : 

(a)  increase  the  objectivity  of  performance  appraisals  by 
utilizing  the  same  automated  management  information 
system  used  for  production  planning  and  budgeting. 
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(b)  include  a  range  of  factors  specifically  geared  to  cover 
the  critical  elements  of  a  shop  foreman's  job,  and 

(a)  encourage  meaningful  feedback  and  communication  between 
foremen  and  their  supervisors. 

The  system  is  comprised  of  10  performance  elements.  Each 
element  can  be  worth  as  much  as  10  points;  therefore,  the 
highest  score  attainable  is  100.  Elements  1  to  5  are 
objective  factors  and  the  performance  factor  (worth  5  of  the 
10  points)  comes  directly  from  the  automated  maintenance  data 
system.  Thus,  this  portion  of  the  rating  is  not  subject  to 
rater  bias  (leniency,  halo,  etc.).  The  rater,  however,  can 
modify  the  objective  performance  score  by  allocating  up  to 
5  more  points  per  element.  This  feature  allows  the  supervisors 
to  give  credit  to  the  individual  whose  objective  score  does 
not  accurately  describe  performance  due  to  factors  beyond  his 
or  her  control.  Elements  6  to  10  are  subjective  factors  rated  by 
the  supervisor  on  a  scale  of  1  to  10  points. 

The  10  elements  incorporated  into  the  appraisal  system  are 
as  follows: 

Objective  Measures 

1 .  Direct  performance  (performance  against  labor  standards) 

2.  Quality  defects  (conformance  with  shop  quality  goals) 

3 .  Ready  for  issue  index  (conformance  with  standard) 

4.  Indirect  budget  performance  (actual  vs.  projected) 

5.  Sick  leave  control  (conformance  with  goal) 

Subjective  Measures 

6.  Schedules 

7.  Safety 

8.  Personnel  Management 

9.  Personal  Characteristics  and  Leadership 

10.  Equal  Employment  Opportunity  Performance 

During  each  appraisal  period,  supervisors  review  the  perfor¬ 
mance  of  foremen  for  the  previous  period  and  establish  specific 
measurable  goals  for  elements  1  to  5  for  the  next  reporting 
period. 


Critique.  This  system  illustrates  the  successful  integration 
of  efficiency  and  effectiveness  measures  in  an  operational 
measurement  system.  With  respect  to  a  work  center,  the 
criteria  measured  are  comprehensive  and  mission  oriented. 


The  system  effectively  combines  objective  measures  generated  by 
existing  automated  maintenance  data  systems  with  subjective 
ratings  of  important  performance  dimensions.  The  use  of 
objective  data  avoids  the  problems  of  self-report  measures 
and  helps  counteract  rater  bias,  such  as  "inflation"  of  per¬ 
formance  ratings.  The  correction  factors  applied  to  the 
objective  measures  allow  evaluators  to  offset  deficiencies 
in  the  objective  measures.  This,  coupled  with  the  fact  that 
people  being  rated  participated  in  its  development,  will 
likely  cause  foremen  to  view  the  system  as  fair  and  make  it 
acceptable  to  them.  Because  it  relies  on  existing  data,  the 
system  is  cost-effective. 

Disadvantages  of  the  system  as  a  productivity  measurement  method 
are  that  it  lacks  measures  of  organzational  impact  and  it  fails 
to  quantify  some  factors  that  could  be  quantified  (safety, 
schedule  compliance,  aspects  of  personnel  management,  etc.). 
Despite  these  objections,  the  approach  demonstrates  quite 
well  how  efficiency  and  effectiveness  can  be  integrated  into 
an  operational  measurement  system  at  the  work  group  level. 

Some  compromises  are  inevitable  in  such  an  operational  system. 
This  approach  represents  an  excellent  example  of  an  integrated 
measurement  method. 

For  evaluation  purposes,  the  system  is  very  useful.  It  yields 
an  overall  performance  score,  individual  performance  dimension 
scores,  and  corrected  and  uncorrected  scores  on  the  objective 
factors.  It  is  not  obvious  which  would  provide  the  most  use¬ 
ful  criteria;  however,  this  could  be  determined  empirically. 

The  system  provides  very  useful  data  for  evaluative  purposes. 
Application  of  the  method  to  Air  Force  organizations  could 
be  made  where  there  is  an  employee-hour  reporting  system  (e.g., 
maintenance) . 

Impact  of  Measurement  on  Productivity 

Productivity  measurement  is  sometimes  viewed  by  managers  as  a  passive 
process  of  recording  the  results  produced  by  the  organization.  Certainly 
measurement  does  fulfill  this  purpose.  Measurement  also  has  a  more 
active  impact.  What  the  organizaion  measures  is  often  viewed  by  employees 
as  synonymous  with  what  the  organization  views  as  important.  Duerr  (1974) 
describes  an  example  of  a  productivity  measurement  system  which  encouraged 
managers  to  engage  in  counter-productive  behavior.  The  company  compared 

each  of  its  12  bottling  plants  monthly  in  terms  of  their  operating 
efficiency  defined  as: 

_  .  .  Number  of  Bottles  Produced 

Operating  Efficiency  =  ~ - — - — : — — - r - rr  ,  , 

Minutes  of  Work  Time  Available  x 

Line  Speed  in  Bottles  per  minute 

Plant  scores  were  posted  monthly  on  each  bottling  line.  Upon  investigat¬ 
ing,  the  researcher  discovered  that  the  system  provided  incentives  for 
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managers  to  keep  improvement  ideas  hidden  from  managers  in  other  plants, 
to  keep  inaccurate  records  of  machine  downtime,  to  "adjust"  actual 
physical  inventory  values,  and  to  incur  costs  in  unmeasured  performance 
areas  to  make  the  operating  efficiency  look  good.  Reliance  on  a  single 
measure  which  was  used  as  a  basis  for  rank  ordering  plants  clearly  had 
counter  productive  effects  (Duerr,  1974) .  Measurement  in  this  organiza¬ 
tion  coupled  with  unfortunate  uses  of  measurement  data  led  to  increased 
costs  rather  than  increased  productivity. 

Interviews  with  managers  in  the  Air  Force  revealed  examples  similar 
to  that  described  in  the  bottling  factories.  Consider  the  following 
illustration  of  the  impact  of  measurement  on  the  behavior  of  Air  Force 
personnel  which  was  supplied  by  a  management  analyst. 

We  track  many  things.  Take  dental  appointments  by  the 
military,  for  example.  If  the  word  gets  out  that  our 
office  is  going  to  present  to  the  commander  data  con¬ 
cerning  missed  dental  appointments,  "no-shows,"  I  can 
show  you  on  our  charts  that  when  we  announce  we  are 
tracking  this  there  will  be  a  drastic  decrease  in  "no 
shows."  Everybody  starts  making  their  dental  appoint¬ 
ments.  So  they  make  their  appointments  for  a  while, 
then  that  chart  will  start  edging  up  again.  Then  we 
hit  the  item  again  and  it  comes  down.  We  select  the 
items  by  those  that  are  getting  out  of  tolerance.  But 
if  Col.  "X"  hears  we  are  going  to  track  "no-shows,"  he 
lets  his  squadron  commanders  know,  they  let  their 
sergeants  know  and  all  of  a  sudden  everyone  is  showing 
up  for  dental  appointments. 

Frequently,  productivity  measurement  relies  on  self-report  measures. 
When  there  are  incentives  to  "look  good"  on  the  measure,  then  self- 
reporting  procedures  have  serious  weaknesses.  This  was  illustrated  in 
the  following  comments  by  the  manager  of  an  accounting  organization. 

When  the  criterion  is  the  number  of  audit  reports 
issued,  what  we  get  is  a  lot  of  short  reports.  When 
the  criterion  is  the  the  number  of  findings,  we  get  a 
lot  of  nit-picky  findings.  When  we  combine  them  and 
count  the  number  of  reports  and  the  number  of  findings 
we  get  a  lot  of  nit-picky  reports. 

The  following  quote  illustrates  how  the  indicators  tracked  by  a 
management  information  system  can  subvert  the  purpose  it  was  designed  to 
accomplish. 

The  program  was  carried  to  such  extremes  that  it 
prevented  the  crew  evaluation  and  training  process. 

The  objective  of  crew  training  was  to  fly  an  instru¬ 
ment  landing  system  (ILS)  approach.  Under  the  Manage¬ 
ment  Information  System  (MIS)  the  objective  became  that 
of  flying  more  ILS  approaches  in  a  given  amount  of  flying 
time.  So  people  cut  approaches  short,  used  different 
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procedures,  used  external  radar  to  get  to  the  "fix" 
rather  than  fly  the  instruments.  They  turned  immediately 
after  crossing  the  runway  instead  of  flying  the  proper 
procedures  for  missed  approach.  People  would  come  out 
with  tremendous  numbers  of  ILSs  but  were  they  produc¬ 
tive  in  the  sense  of  improving  their  ability?  The  deci¬ 
sion  that  came  through  is  that  they  were  not. 

This  manager  goes  on  to  identify  the  crux  of  the  issue  regarding 
measurement  and  how  measures  are  used. 

We  quantify  the  quantifiable  and  you  get  what  you  reward 
don't  you?  In  the  case  of  the  MIS  we  reward  what  we 
measure.  You  put  all  those  numbers  on  a  sheet  and  every¬ 
body  starts  focusing  on  the  numbers  which  only  tell  part 
of  the  story. 

The  numbers  only  tell  part  of  the  story!  This  points  out  one  of  the 
most  basic  criticisms  of  productivity  measurement  as  currently  practiced 
in  the  Air  Force.  Reliance  on  one  or  two  measures  to  assess  the  produc¬ 
tivity  of  an  organization  leads  to  the  types  of  unintended  consequences 
reported  above.  Productivity  measurement  should  be  broadened  to  en¬ 
compass  the  types  of  efficiency  and  effectiveness  criteria  discussed 
in  this  report.  The  next  chapters  describe  a  proposed  methodology 
for  developing  such  a  set  of  measures  for  use  in  Air  Force  organizations. 


V.  CONCLUSIONS  AND  IMPLICATIONS  FOR  AIR  FORCK  PRODUCTIVITY  MEASUREMENT 

As  stated  in  Chapter  I,  the  objectives  of  this  report  were  to 
clarify  the  meaning  of  productivity  as  it  applies  to  Air  Force  organi¬ 
zations,  to  assess  existing  productivity  measurement  methods,  and  to 
develop  a  methodology  for  generating  Air  Force  productivity  criteria. 
Chapters  II  through  IV  focused  on  the  first  two  objectives.  This 
chapter  summarizes  the  major  conclusions  emerging  from  the  preceding 
discussion  which  have  implications  for  the  third  project  objective, 
development  of  a  productivity  measurement  methodology  for  use  in  Air 
Force  organizations. 

Conclusions  From  Previous  Research 

Based  on  material  reviewed  in  the  preceding  chapters,  six  points 
were  found  to  be  important  in  guiding  the  development  of  a  productivity 
measurement  methodology  for  use  in  Air  Force  organizations. 

1 .  The  definition  of  productivity  for  Air  Force  organizations 
should  incorporate  the  concepts  of  both  efficiency  and 
effectiveness  (Balk,  1975;  Coulter,  1979;  Engle,  1979; 
Letzkus,  1973;  National  Center  for  Productivity  and  Quality 
of  Working  Life,  1977).  Productivity  measurement  programs 
that  monitor  only  efficiency  may  lead  to  the  conclusion 
that  improvements  are  being  made  when, in  fact,  efficiency 
improvements  are  being  made  at  the  expense  of  effectiveness 
as  measured  by  indicators  of  quality,  safety .customer  sat¬ 
isfaction,  or  future  performance  capability.  On  the  other 
hand,  programs  that  monitor  only  effectiveness  may  fail  to 
consider  costs  associated  with  achieving  the  desired  levels 
of  effectiveness,  and  therefore  may  contribute  to  inefficient 
use  of  resources. 

2 .  An  organizational  productivity  measurement  scheme  should 
include  multiple  measures  of  both  efficiency  and  effectiveness 
(Herrick,  1975;  Macy  &  Mirvis,  1976;  National  Center  for 
Productivity  and  Quality  of  Working  Life,  1977),  Organizations 
have  multiple  goals  and  mission  statements  have  many  facets. 
Therefore,  assessments  of  organizational  efficiency  and 
effectiveness  must  be  multidimensional  to  assure  that  these 
multiple  facets  are  covered. 

Since  measurement  can  affect  performance,  failure  to  measure 
a  critical  aspect  of  mission  accomp1 ishment  may  be  a  signal 
to  organizational  members  to  divert  time,  attention,  and  re¬ 
sources  to  those  mission  areas  that  are  measured  in  an  effort 
to  "look  good." 

As  a  guide  to  developing  a  comprehensive  set  of  measures,  the 
criterion  categories  listed  in  Table  7  may  provide  a  useful 
point  of  departure  . 


Table  7 


Criterion  Categories  to  Consider  in  Developing  a  Set  of 
Productivity  Measures  for  an  Air  Force  Organization 


Criterion  Category 


Potential  Measures 


Efficiency 


Effect iveness 

Quality  of  Performance 


Timeliness 


Performance  Impact  on 
Customers 


Performance  Impact  on  the 
Organization  Members 


Output  A  (expressed  in  physical 

units) _ 

Inputs  used  to  produce  Output  A 
(expressed  in  constant  dollars) 

(Repeat  for  key  outputs  B,  C,  D, 
etc . ,  and  compare  with  previous 
period,  budget,  or  goals) . 


Number  of  errors,  amount  of  rework, 
amount  of  scrap,  number  of  rejects 
or  returned  items,  etc.,  compared 
to  budgets,  standards,  or  goals. 
(May  be  done  for  each  product  or 
service.) 

Response  time,  processing  time, 
schedule  reliability,  etc.,  com¬ 
pared  to  standards  or  goals. 

Effects  (positive  or  negative)  on 
the  user  organizations' s  mission 
performance,  customer  complaints, 
responsiveness  to  complaints, 
customer  attitudes,  etc.,  compared 
to  previous  period  or  goal. 

Morale,  training  quality,  safety 
maintenance  and  availability  of 
tools  and  equipment,  health  status, 
disciplinary  actions,  etc. 


3 .  A  set  of  efficiency  and  effectiveness  measures  should  have  as 
many  as  possible  of  the  following  characteristics  (Hurst, 

1980;  Joint  Financial  Management  Improvement  Program,  1977; 
Kearney,  A.  T.,  Inc.,  1978): 

a.  Completeness .  The  set  of  measures  should  comprehensively 
cover  the  significant  facets  of  the  organization's  mission 
and  the  resources  expended  in  mission-related  activities. 
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b.  Comparability.  Measures  should  be  comparable  from  one 
time  period  to  another.  To  make  meaningful  longitudinal 
comparisons  of  productivity  for  organizational  units,  the 
measures  should  remain  applicable  through  time.  In 
general,  it  is  not  cost-effective  to  include  measures  of 
non-recurring  workloads  or  functions  in  a  productivity 
measurement  system  that  will  be  used  to  track  an  organi¬ 
zation's  productivity  over  time. 

c.  Input  coverage.  Outputs  used  in  the  efficiency  measures 
should  comprehensively  cover  the  relevant  inputs  used  by 
organization.  For  example,  if  labor  is  the  primary  in¬ 
put,  the  output  measures  should  comprehensively  cover  the 
results  obtained  from  all  employee  hours  worked.  To 

the  extent  that  mission-related  transformations  of 
input  are  not  reflected  in  the  output  measures,  efficiency 
will  be  understated.  On  the  other  hand,  output  measures 
should  not  reflect  transformations  of  input  that  are  not 
mission  related. 

d.  Compatability  with  existing  data  sources.  In  general, 
measures  should  be  generated  which  make  use  of  existing 
data  sources  rather  than  require  collection  of  new  data. 

This  is  true  because  it  reduces  the  cost  of  measurement 
and  increases  the  likelihood  that  measures  will  be  im¬ 
plemented. 

e.  Cost-effectiveness.  The  benefits  derived  from  a  set  of 
measures  must  exceed  the  costs  associated  with  collecting 
the  data  and  preparing  management  productivity  reports. 

f.  Consistent  across  organizations.  To  be  useful  for  research, 
measures  must  be  consistent  across  organizations.  While 
measures  may  vary  across  organizations  performing  dif¬ 
ferent  functions,  the  most  useful  set  would  be  relatively 
invariant  across  organizations  performing  the  same  functions. 

g.  Acceptable  to  organization  members.  Managers  and  members 
of  the  organization  whose  work  output  is  being  measured 
must  accept  the  measures  as  being  fair  and  useful. 

Acceptance  of  the  productivity  measures  is  increased  if 
individuals  whose  work  is  being  measured  and  managers  who 
will  use  the  measurement  data  participate  in  its  develop¬ 
ment. 

4 .  Individual  measures  in  a  set  of  productivity  measures  are  most 

useful  if  they  possess  the  following  features  (Hurst,  1980; 

Joint  Financial  Management  Improvement  Program,  1977;  Kearney, 

A.  T. ,  Inc. ,  1978) : 

a.  Val idity .  Individual  measures  should  accurately  reflect 
changes  in  the  mission-related  aspect  of  efficiency  or 
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effectiveness  being  measured.  Valid  measures  are 
relatively  free  of  measurement  biases  such  as  omission 
of  pertinent  elements  of  performance  (deficiency) .  includ¬ 
ing  extraneous  informaton  unrelated  to  productivity  (con¬ 
tamination)  or  combining  non-equivalent  elements  in  con¬ 
structing  input  or  output  values  (distortion) . 

b.  Uniqueness .  The  measure  should  account  for  some  aspect 
of  productivity  that  is  important  to  the  organization  and 
not  measured  by  any  other  indicator  in  the  measurement 
set. 

c.  Under standability.  The  relationship  between  the  level  of 
performance  and  the  measure  should  be  understood  both  by 
managers  who  will  use  the  data  as  well  as  the  organiza¬ 
tional  members  whose  work  outputs  are  being  measured. 

d.  Controllability.  Members  of  the  organization  being 
measured  should  be  able  to  influence  the  organizational 
output  being  measured.  Individuals  should  not  be  assessed 
and  held  accountable  for  results  that  are  outside  their 
control . 

e.  Reliablity.  Repeated  reapplication  of  the  indicator  to 
the  same  level  of  organization  performance  should  produce 
the  same  measurement  or  score.  Measures  should  also  be 
based  on  accurate  and  reliable  source  data. 

5 .  For  most  Air  Force  applications,  Form  III  efficiency  measures 
which  employ  outputs  expressed  in  physical  units  and  inputs 
expressed  as  dollar  costs  are  most  useful  (Felsinger,  1976)  . 
Forms  of  the  efficiency  ratio  that  make  use  of  dollar  values  of 
output  are  geneially  not  applicable  to  Air  Force  organizations 
because  of  the  difficulty  of  establishing  the  "market  value" 

of  outputs.  Dollar  costs  of  inputs  however,  can  usually  be 
obtained  and  will  provide  a  common  denominator  for  use  in 
aggregating  inputs.  This  facilitates  the  use  of  total  factor 
productivity  ratios  as  opposed  to  less  meaningful  partial 
measures . 

6 .  Efficiency  and  effectiveness  measures  should  be  developed  for 
the  key  facets  of  mission  performance  (Duerr,  1974; 
Mollenhoff,  1977).  For  any  organization,  there  is  virtually 
an  unlimited  number  of  possible  productivity  measures,  but 
the  number  of  measures  developed  should  be  kept  to  a  manage¬ 
able  number.  This  requires  the  establishment  of  priorities 
among  the  various  facets  of  performance  and  the  development 

of  measures  only  for  the  areas  judged  to  be  key  results  areas. 

7 .  Measurement  activities  in  Air  Force  organizations  should 
make  use  of  a  measurement  coordinator  (National  Center  for 
Productivity  and  Quality  of  Working  Life,  1978).  Experience 
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with  the  Total  Performance  Management  process  suggests  that  the 
development  of  a  productivity  measurement  program  is  most 
effective  when  a  measurement  coordinator  is  involved.  In  the 
Air  Force,  such  an  individual  may  come  from  a  staff  or  support 
organization.  The  role  of  the  coordinator  is  to  organize 
the  process,  insure  its  technical  soundness  and  to  be  the 
catalyst  who  assures  that  the  process  is  completed.  Ideally, 
the  coordinator  will  have  had  experience  in  working  as  a  qroup 
facilitator  or  trainer  and  have  knowledge  of  measurement 
procedures  in  either  industrial  engineering  or  behavioral 
science.  Where  the  coordinator  lacks  this  expertise,  access 
to  individuals  with  these  skills  and  knowledge  should  be 
arranged. 

Implications  for  Methodology  Development 

Implications  for  developing  a  productivity  measurement  methodology 
for  use  in  Air  Force  organizations  may  be  derived  directly  from  the 
conclusions  in  the  previous  section.  Five  implications  seem  particularly 
relevant.  Figure  6  depicts  the  relationship  between  the  conclusions 
presented  earlier  and  the  five  implications  for  methodology  development. 


Item  Identification  of  Conclusions 


Implications 


Which  Lead  to  the  Implication 


4  c,  d 


1. 


Employees  should  participate 
in  the  measurement  process 


2. 


Management  should  decide 
what  facets  of  performance 
will  be  measured. 


3  d,  e 

1,  2,  3,  4,  5,  6,  7 


3.  Existing  data  should  be  used. 


4.  A  measurement  coordinator  is 
needed . 


1,  2,  3,  f 


5. 


The  methodology  must  generate 
criteria  which  are  useful  for 
research  purposes. 


Figure  6.  Process  of  Deriving  Implications  from  Conclusions 


Employees  should  participate  in  the  measurement  process. 

The  measurement  process  may  impact  efficiency  and  effectiveness 
positively  or  negatively  (Duerr,  1974) .  If  measures  are  per¬ 
ceived  by  organizational  members  as  being  unfair  or  inaccurate, 
and  if  they  are  used  in  negative  rather  than  positive  ways, 
then  they  may  adversely  affect  organizational  productivity. 
However,  if  measures  are  viewed  as  fair  and  reasonable  and 
are  used  to  provide  meaningful  feedback  to  members,  then  they 


will  likely  have  a  positive  impact  on  productivity.  Therefore, 
to  help  insure  that  the  impact  of  the  measurement  process  on 
performance  will  be  positive,  individuals  being  measured 
should  have  an  opportunity  to  participate  in  the  process  of 
developing  the  measures.  Participation  can  be  expected  to 
improve  worker  acceptance  of  the  resulting  measurement  process, 
take  advantage  of  the  worker's  detailed  knowledge  of  the  work 
process,  and  therefore  enhance  the  quality  of  measures  and  the 
productivity  of  the  organization  (Adam,  Hershauer  and  Ruch, 
1978;  Balk,  1975;  Taylor,  1979) . 

2 .  Management  should  decide  which  facets  of  performance  will  be 
measured .  There  are  many  possible  indicators  of  efficiency 
and  effectiveness.  A  methodology  to  generate  productivity 
criteria  should  attempt  to  restrict  the  set  of  measures 
developed  to  those  that  are  most  important  for  organizational 
decision  making.  Management  of  an  Air  Force  organization  is 
responsible  for  establishing  priorities  and  for  judging  the 
extent  to  which  activities  contribute  to  mission  accomplish¬ 
ment  (AFM  25-1,  1964).  Therefore,  the  selected  methodology 
should  provide  management  with  the  opportunity  to  determine 
which  aspects  of  organizational  efficiency  and  effectiveness 
are  to  be  measured. 

3.  Use  existing  data  if  possible.  Any  effort  to  measure  produc¬ 
tivity  in  an  Air  Force  organization  should  make  use  of  exist¬ 
ing  data  and  information  systems  rather  than  require  the 
collection  of  new  data  (Air  Force  Worldwide  Productivity 
Conference ,  1978) .  This  implication  takes  into  account  the 
fact  that  new  data  collection  is  expensive  and  should  only  be 
carried  out  when  the  value  of  the  data  is  sufficient  to 
justify  the  cost.  This  principle,  however,  should  not  be 
allowed  to  divert  the  process  away  from  potentially  important 
key  results  areas  simply  because  of  a  lack  of  data.  On  the 
other  hand,  the  measurement  process  should  not  allow  existing 
data  to  lead  to  the  inclusion  of  trivial  or  misleading 
measures . 

4.  Measurement  coordinator  needed.  The  measurement  methodology 
should  be  designed  on  the  assumption  that  a  measurement  co¬ 
ordinator  will  direct  its  implementation.  The  coordinator 
should  "steer"  the  measurement  process  sufficiently  to  insure 
that  the  measures  defined  are  consistent  with  the  criteria 

of  technical  adequacy  discussed  previously  in  conclusions 
number  3  and  4.  As  a  result  of  this  responsibility,  the 
coordinator  should  have  an  understanding  of  the  strengths  and 
weaknesses  of  the  productivity  measurement  approaches 
discussed  in  Chapter  IV. 

5.  Methodology  must  generate  research  criteria.  The  implica¬ 
tions  have,  to  this  point,  focused  on  the  need  for  develop¬ 
ing  . i  methodology  which  will  yield  productivity  criteria 
which  can  be  used  for  a  single  organization.  In  addition 
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to  this  need,  the  methodology  must  generate  criteria  which 
are  suitable  for  research  in  multiple  organizations  that 
perform  similar  work.  Generalizable  criteria  are  necessary 
to  assess  the  effects  of  organizational  change  programs  on 
productivity  using  cross-sectional,  as  opposed  to  longitudinal 
research  designs.  Developing  such  criteria  is  a  major  goal 
of  the  present  research  and  of  the  methodology  described  in 
Chapter  VI . 
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VI.  METHODOLOGY  FOR  GENERATING  PRODUCTIVITY  INDICATORS 


Based  on  the  preceding  review  of  productivity  measurement  and 
improvement  techniques,  this  chapter  describes  a  new  approach  for 
generating  productivity  indicators  in  an  Air  Force  organization.  The 
methodology  is  intended  to  be  sufficiently  general  to  be  applicable  to 
any  functional  area  and  to  any  organizational  level,  including  sections, 
branches,  squadrons,  or  wings.  The  proposed  procedures  represent  a 
synthesis  of  previous  research  (e.g.,  Adam,  Hershauer,  &  Ruch,  1978; 

Balk,  1975;  Mollenhoff,  1977;  Ohio  State  Productivity  Research  Group, 

1977) .  In  tailoring  the  approach  to  the  Air  Force  environment,  modifica¬ 
tions  in  previously  developed  methodologies  were  required.  In  particular, 
these  modifications  involved  derivation  of  key  results  areas  (KRA's) 
from  mission  statements,  formation  of  groups,  sequencing  of  group 
activities  to  involve  commanders  at  critical  stages  of  the  process, 
and,  developing  procedures  for  generalizing  indicators  across  organiza¬ 
tions. 

Overview  of  the  Measurement  Procedure 


The  methodology  has  five  phases:  Background,  Definition  of  Key 
Results  Areas  (KRAs) ,  Definition  of  Indicators  and  Data  Sources, 
Generalization  of  Indicators,  and  Implementation. 

1.  Background .  The  background  phase  involves  the  initial  decision  to 
measure  productivity  in  a  particular  organization,  selection  and 
orientation  of  measurement  coordinators,  familiarization  of  coordin¬ 
ators  with  the  organization  to  be  measured,  and  development  of  a 
description  or  "model"  of  the  target  organization  in  systems  terms. 

2.  Definition  of  Key  Results  Areas  (KRAs) .  In  this  phase,  a  group 
including  the  top  two  levels  of  management  of  the  target  organization 
(Group  A)  is  convened.  The  process  involves  a  consideration  of  the 
mission  of  the  organization  and  uses  structured  group  procedures  to 
break  the  mission  into  categories  that  describe  the  organization's 
principal  intended  accomplishments.  These  categories  are  called 

key  results  areas  (KRAs) . 


3 .  Definition  of  Indicators  and  Data  Sources.  This  phase  involves  con¬ 
vening  a  group  of  individuals  who  are  drawn  from  the  second  level  of 
management  and  their  immediate  subordinates  (Group  B) .  Using  a  structur¬ 
ed  group  process  similar  to  that  used  in  Phase  2,  Definition  of  Key 
Results  Areas,  this  group  is  asked  to  generate  and  prioritize  specific 
indicators  for  each  of  the  key  results  areas  developed  above.  In 
addition.  Group  B  is  asked  to  suggest  additional  key  results  areas  if 
they  feel  that  important  areas  were  omitted  by  Group  A.  Indicators  will 
also  be  generated  for  any  new  KRA's  recommended  by  Group  B. 

Once  these  indicators  are  developed,  a  third  group  (Group  C)  is 
formed  by  drawing  members  from  both  Group  A  and  B.  In  a  discussion  mode. 
Group  C  reviews  the  products  generated  by  Groups  A  and  B,  and  redefines  any 


items  needing  clarification  or  additional  definition.  Once  satisfied 
with  the  definitions  of  key  results  areas  and  the  specific  indicators. 

Group  C  identifies  existing  sources  of  data  for  each  indicator.  Indicators 
for  which  no  data  is  currently  available  are  identified  and  pro¬ 
cedures  are  proposed  for  obtaining  the  required  data.  At  the  end  of  Phase 

3.  a  set  of  indicators  will  have  been  defined  along  with  the  sources  of 
data  required  to  implement  the  measurement  process. 

4.  Generalization  of  Indicators  (Optional  Phase) .  If  the  indicators  are 
to  be  used  only  within  the  organization  involved  in  generating  them. 

Phase  4  is  not  required.  However,  if  the  methodology  is  to  be  used  to 
develop  a  common  set  of  productivity  indicators  for  use  across  organiza¬ 
tions  within  a  functional  area,  then  Phase  4  is  required.  The  purpose 
of  this  phase  is  to  develop  generalizable  criteria  that  can  be  applied 
to  multiple  organizations.  This  phase  would  be  required  in  developing 
research  criteria  applicable  to  a  number  of  similar  organizations. 

Stated  in  the  most  simple  terms,  Phase  4  involves  repeating  abbreviated 
versions  of  Phase  1  to  3  with  organizations  B,  C,  D,  etc.,  until  a 
recurring  set  of  indicators  is  obtained  which  can  be  consistently  applied 
across  the  organizations. 

5.  Implementation.  Once  the  indicators  have  been  developed.  Phase  5 
involves  actual  measurement  of  the  productivity  of  the  target  organiza¬ 
tion  (s)  .  Implementation  strategies  will  differ  considerably  from 
organization  to  organization  and  any  detailed  discussion  of  a  particular 
implementation  is  beyond  the  scope  of  this  report. 

Description  of  the  Measurement  Procedure 

This  section  will  describe  each  of  the  major  steps  in  the  procedure 
for  generating  productivity  criteria.  The  discussion  will  be  organized 
according  to  the  previously  described  phases  of  the  methodology,  if  the 
purpose  of  measurement  is  to  develop  productivity  indicators  for  a 
single  organization,  then  the  process  should  begin  with  Phase  1.  However 
if  the  purpose  is  to  develop  generalizable  indicators  for  research  pur¬ 
poses,  then  one  should  start  the  process  with  Phase  4. 


Phase  1.  Background  Phase 

1.1  Decision  to  measure  productivity.  The  procedure  starts  with 
a  decision  to  measure  the  productivity  of  an  organization. 

The  initial  purpose  for  measurement  has  implications  for  the 
procedures  followed  and  for  the  outcome.  There  are  many 
reasons  why  productivity  measurement  may  be  undertaken.  For 
example  a  unit  commander  may  be  having  problems  and  desires 
to  call  in  an  "outside"  consultant  (e.g.,  Base  Management 
Engineering  Team  or  a  consulting  team  from  the  Leadership  and 
Management  Development  Center  at  Maxwell  AFB)  to  help  the 
organization  improve  its  performance.  As  a  part  of  this 
process,  either  the  manager  or  the  "consultant"  may  decide 
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that  productivity  indicators  should  be  developed  prior 
to  the  intervention  to  determine  if  the  organizational  change 
program  "works"  and  has  no  unintended  negative  side  effects. 

The  Air  Force  productivity  program  also  acts  as  a  stimulus  to 
local  commanders  to  initiate  productivity  improvement  efforts. 
Under  this  program,  organizations  are  asked  to  report  pro¬ 
ductivity  indicators  to  higher  headquarters.  As  a  part  of  his 
or  her  productivity  plan,  a  commander  may  decide  to  develop 
a  comprehensive  set  of  indicators  that  will  have  utility  for 
managing  the  organization,  for  planning  and  budgeting  and  for 
satisfying  the  requirements  of  the  plan.  Finally,  an  Air 
Force  laboratory  conducting  research  on  productivity  enhance¬ 
ment  efforts  may  use  the  methodology  for  generating  criteria 
for  assessing  the  impact  of  organizational  change  programs  on 
productivity.  These  are  just  a  few  of  the  reasons  for  initiat¬ 
ing  the  measurement  procedures  described. 

1 . 2  Select  measurement  coordinators.  In  order  for  the  procedures 
to  be  adequately  followed,  it  is  necessary  to  select  at  least 
one,  (preferably  two)  measurement  coordinators.  The  use  of 
two  coordinators  helps  guard  against  the  introduction  of  bias 
and  is  most  effective  for  conducting  the  group  process  proce¬ 
dures.  Coordinators  selected  should  not  be  members  of  the 
target  organization  or  in  its  regular  chain  of  command.  They 
should  have  good  verbal  skills,  some  experience  in  group 
facilitation,  and  some  familiarity  with  the  basic  concepts  of 
productivity  and  productivity  measurement.  The  preceding 
chapters  of  this  report  would  provide  coordinators  a  reason¬ 
able  introduction  to  the  subjects  of  productivity  and  pro¬ 
ductivity  measurement. 

1.3  Familiarization  with  the  organization.  Once  selected,  the 
measurement  coordinators  should  familiarize  themselves  with 
the  organization  under  study.  This  familiarization  process 
might  begin  with  a  meeting  between  the  coordinators  and  key 
members  of  the  target  organization  and  its  parent  unit.  In 
this  initial  meeting,  coordinators  should  be  introduced  to 
members  of  the  organization,  and  the  purpose  of  the  measurement 
program  should  be  discussed. 

Following  this  initial  meeting,  it  is  helpful  for  the  co¬ 
ordinators  to  spend  some  time  with  the  commander  of  the 
parent  unit  of  the  target  organization.  This  session  can 
shed  considerable  light  on  the  unit's  history,  current  status, 
mission,  and  organizational  structure  and  provide  coordinators 
an  overview  of  the  target  organization's  role  in  its  laraer 
organizational  context. 

During  this  visit  to  the  organization,  coordinators  should 
obtain  relevant  documentation  describing  the  mission,  organiza¬ 
tion,  manning  posture,  types  of  work  performed,  and  the  types 
of  management  analysis  data  currently  being  collected.  At 
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this  stage  of  the  process  coordinators  should  try  to  avoid 
immersion  in  detail.  Instead,  they  should  focus  on  the 
organization  as  a  system.  In  so  doing, they  should  attempt  to 
determine  significant  inputs  and  outputs,  as  well  as  other 
significant  organizations  in  the  target  organization's  environ¬ 
ment. 

1.4  Define  organizational  boundaries.  In  the  Air  Force,  boundaries 
between  organizational  units  are  usually  quite  clear.  However, 
if  the  specific  boundaries  of  the  target  organization  are  un¬ 
clear,  the  coordinators  should  define  explicitly  what  is  within 
the  scope  of  the  measurement  effort  and  what  is  outside  that 
scope.  For  example,  if  the  target  organizaton  is  a  wing-level 
organization  which  also  operates  a  base,  then  decisions  have 

to  be  made  as  to  whether  tenant  units  on  the  base  that  are 
not  part  of  the  wing  organization  are  to  be  included.  Some  of 
the  issues  to  be  resolved  include  the  following: 

a.  What  personnel  are  considered  within  the  organization  (e.g., 
reservists,  civilians,  uniformed  military) ? 

b.  Where  do  customer  interfaces  occur? 

c.  What  is  the  nature  of  the  service  demands,  timing 
and  location? 

d.  Are  people  on  detail  to  be  included? 

e.  What  levels  of  management  and  what  functional  groups  are 
to  be  included  ? 

f.  What  physical  plant  and  resources  are  to  be  included? 

g.  With  what  other  organizational  units  does  the  target 
organization  interface? 

1.5  Construct  organizational  diagram.  To  conclude  Phase  1,  and 
to  provide  input  for  the  group  procedures  in  Phase  2,  the  co¬ 
ordinators  should  construct  a  systems  diagram  of  the  target 
organization.  This  diagram  should  depict  major  elements 
pertaining  to  the  organization's  structure  and  manning  posture. 
It  should  depict  the  overall  flow  of  work  and  show  major  in¬ 
puts,  outputs,  and  significant  interrelationships  with  other 
organizations . 

Phase  2.  Definition  of  Key  Results  Areas 

2.1  Form  management  Croup  A.  Phase  2  begins  with  the  coordinators 
identifying  individuals  to  serve  as  members  of  Group  A. 

Since  the  major  function  of  this  group  is  to  identify  the  key 
results  areas  for  the  target  organization,  its  members  should 
be  in  a  position  to  make  knowledgeable  determinations  of  this 
sort.  Generally,  Group  A  will  consist  of  members  of  the  target 
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organization's  management  and  working  supervision  and  represen¬ 
tatives  of  the  parent  organization.  For  a  guide  to  determining 
membership  of  Group  A,  see  Figure  7  . 

Figure  7  lists  the  steps  in  sequential  order  that  should  be 
followed  in  forming  Groups  A,  B,  and  C.  Steps  1  to  4  place 
the  target  organization  in  the  context  of  its  organizational 
hierarchy.  Once  Step  4  has  been  accomplished,  forming  groups 
as  described  in  Step  6  is  straightforward.  As  pointed  out  in 
Step  7,  when  dealing  with  an  organization  at  the  bottom  of 
the  organizational  hierarchy,  Groups  A  and  B  will  be  the  same 
group.  In  organizations  that  provide  services  to  the  public, 
it  may  also  be  helpful  to  include  representatives  from  customer 
groups  or  organizations. 


The  principal  concern  in  forming  Group  A  should  be  to  include 
policy  and  decision  makers  who  have  knowledge  of  all  facets  of 
the  organization's  performance.  The  optimum  group  size  should 
be  6  to  12  participants. 

2.2  Orient  Group  A.  Once  the  group  members  are  selected,  coordi¬ 
nators  should  announce  a  group  meeting  following  the  appro¬ 
priate  communication  channels.  The  orientation  has  several 
purposes.  It  should  attempt  to  gain  the  support  of  the 
members  for  the  activity,  it  should  inform  members  of  the 
benefits  of  measurement  for  the  organization  and  for  the 
Air  Force,  and  it  should  acquaint  them  with  fundamentals  of 
efficiency  and  effectiveness  measurement  and  the  approach  to 
be  followed.  In  effect,  this  session  should  attempt  to  sell 
participants  on  the  approach  in  order  to  obtain  their  en¬ 
thusiastic  participation,  and  it  should  guide  them  toward  the 
goal  of  identifying  key  results  areas. 

2.3.  Generation  of  Key  Results  Areas  (KRAs)  -  The  concept  of 

"key  results  areas,"  or  KRA,  borrows  from  Mollenhoff  (1977). 
In  his  conception,  KRAs  are  the  results  that  individuals  in 
an  organization  are  paid  to  achieve.  This  same  concept  might 
be  called  organizatonal  objectives  by  others.  In  an  Air  Force 
organization,  KRAs  represent  a  way  to  break  the  unit's  mission 
statement  into  operational  parts  that  are  amenable  to  measure¬ 
ment  . 

The  process  of  generating  KRAs  makes  use  of  a  structured  group 
process  called  the  nominal  group  technique  (Delbecq,  Van  de 
Ven,  &  Gustafson,  1975) .  The  process  has  the  following  steps: 
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Steps 


Example 


1.  Identify  target  organization 


Squadron 


2.  Identify  next  highest  level 
organization 


Wing 


3.  Identify  next  two  levels  below 
target  organization.  The  level 
below  the  lowest  organizational 
level  with  a  supervisor  would  be 
called  "organizational  members." 

4.  List  these  in  hierarchical  order 


Branch  and  Section 


1 .  Wing 

2 .  Squadron3 

3 .  Branches 

4.  Sections 


5.  If  the  next  level  below  the  target 
organization  is  "organizational 
members" ,  go  to  Step  7 

6.  If  three  organizational  levels  can 
be  identified  then  determine  groups 
as  follows: 

a.  Group  A  consists  of  representatives 
of  levels  1,  2  and  3 

b.  Group  B  consists  of  representatives 
of  levels  3  and  4 

c.  Group  C  consists  of  selected 
individuals  from  Groups  A  and 
B 


Group  A 

1.  Wing 

2 .  Squadron3 

3 .  Branches 

Group  B 

3 .  Branches 

4.  Sections 


7.  If  only  three  levels  can  be  defined  then 
Groups  A  and  B  are  the  same  group. 


Groups  A  &  B 

1.  Branch 

2 .  Section3 

3 .  Members 


1 

j 


Figure  7*  Guide  to  Determining  Group  Membership 
3Denotes  target  organization  for  measurement  procedure 
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2.3.1.  Silent  generation.  Members  are  asked  to  independently 
list  on  a  sheet  of  paper  the  KRAs  for  the  organization. 
Coordinators  might  make  this  process  clearer  by  asking 
individuals  to  write  their  answers  to  the  question 
"What  results  are  the  members  of  this  organization  paid 
to  achieve."  Allow  5  to  10  minutes  for  the  listing. 

2.3.2.  Round-robin  listing.  Once  sufficient  time  has  been 
allowed  and  each  member  is  ready  to  proceed,  coordinators 
will  proceed  around  the  group  in  round-robin  fashion 
asking  members  to  share  one  of  their  KRAs.  This  process 
is  repeated  until  all  members  have  exhausted  their  lists. 
Items  suggested  by  members  are  listed  on  flip  charts 

by  the  coordinators  without  comment  or  without  dis¬ 
cussion  by  other  members  of  the  group. 

2.3.3.  Clarification .  Following  the  listing,  group  members 
are  given  time  to  review  the  list  and  to  ask  questions 
of  other  members  in  order  to  clarify  the  meaning  of 
particular  items.  In  this  discussion  phase,  it  is 
common  for  members  to  suggest  combinations  of  items, 
groupings  of  two  or  more  under  a  single  heading,  and 
other  revisions.  The  coordinators'  role  is  to  steer 
the  discussion,  to  keep  it  focused  on  the  task,  and  to 
urge  resolution  of  questions  concerning  the  meaning  of 
items.  When  group  members  are  satisfied  with  the  mean¬ 
ing  of  each  of  the  item,  and  when  redundant  or  overlap - 
ing  items  have  been  combined  or  eliminated,  the  procedure 
continues  to  the  next  step. 

2.3.4.  Vote  1 .  Members  are  then  asked  to  select  the  items  from 
the  list  which  they  feel  are  the  most  important  KRAs 
for  the  unit.  The  number  selected  is  arbitrary  and 
depends  on  the  number  of  items  listed  by  the  group. 
Generally,  having  the  group  list  5  to  8  items  is  sat¬ 
isfactory.  Members  are  then  provided  with  index  cards. 
Assuming  that  they  are  asked  to  select  the  top  5  items, 
they  are  asked  to  write  these  five  on  index  cards,  one 
to  a  card.  The  cards  are  then  sorted  in  priority  order. 
Weights  are  assigned  to  items  by  members  as  follows. 

The  most  important  item  is  given  a  "5",  next  most 
important  a  "4",  and  so  on.  Each  member's  set  of  five 
cards  is  then  collected  by  the  coordinators. 

2.3.5.  Tally  results  of  vote  1.  At  this  stage  of  the  process, 
members  are  given  a  break,  and  coordinators  tally  the 
results  of  the  voting.  The  total  number  of  points  re¬ 
ceived  by  each  item  and  the  voting  results  for  each 

item  is  displayed  on  the  flip  charts  by  the  coordinators. 
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2.3.6.  Review  vote  1  results.  Members  then  return  from  the 
break  and  are  asked  to  review  the  voting  results.  Where 
voting  patterns  are  inconsistent,  for  example  in  a  12 
member  group  an  item  may  receive  six  5's  and  six  no 
votes  (i.e.  the  item  was  not  selected  as  one  of  the  top 
5  by  6  members) .  This  indicates  either  an  ambiguous 
item  or  a  major  discrepancy  between  members  regarding 
the  importance  of  a  KRA.  Coordinators  should  question 
members  to  determine  the  reasons  for  such  inconsistent 
voting  and  then  lead  the  group  to  resolve  the  problem. 
If  the  item  is  unclear  and  subject  to  misinterpretation, 
then  it  should  be  revised.  If  there  is  a  legitimate 
disagreement  among  members  regarding  an  item’s  impor¬ 
tance,  then  some  discussion  should  be  allowed.  In  this 
case,  the  role  of  the  coordinators  is  to  insure  that 
this  process  is  one  of  clarification  of  the  issues 

and  not  the  exercise  of  social  pressure.  Discussion 
should  be  allowed  to  continue  until  all  points  of  view 
are  heard,  although  disagreements  may  remain  at  this 
stage . 

2.3.7.  Vote  2 .  This  step  involves  a  repeat  of  the  voting 
process  in  2.3.4. 

2.3.8.  Tally  results  of  vote  2.  Following  voting,  the  co¬ 
ordinators  tally  the  results  and  display  them  on  the 
flip  charts  in  front  of  the  group  as  feedback.  Ideally, 
members  will  remain  while  the  votes  are  being  tallied 

in  order  to  receive  immediate  feedback.  This  completes 
the  nominal  group  process  for  Group  A.  Past  experience 
shows  that  the  total  time  required  for  this  procedure 
is  3  to  4  hours. 

2 . 4  Develop  chart  listing  and  describing  Key  Results  Areas %  To 
conclude  the  second  phase  of  the  methodology,  coordinators 
will  summarize  the  Group  A  activity  on  a  chart  which  depicts 
and  defines  the  KRAs.  The  chart  also  lists  KRAs  in  priority 
order  showing  the  numerical  score  received  by  each  during  the 
final  voting.  This  chart  constitutes  input  into  Phase  3. 

Phase  3.  Definition  of  Productivity  Indices  and  Measurement  Methods 

3.1  Form  Group  B.  Group  B  is  composed  of  individuals  from  two 
organizational  levels  below  the  target  organization.  For  ex¬ 
ample,  if  the  target  organization  is  at  the  squadron  level. 
Group  B  will  include  individuals  from  the  branch  level  (who 
were  also  in  Group  A)  as  well  as  individuals  from  the  section 
or  work  center  level.  See  Figure  7  for  an  illustration  of 
this  procedure. 

3.2  Orient  Group  B.  As  was  the  case  with  Group  A,  Group  B  should 
be  briefed  on  the  purposes  of  the  measurement  effort,  the 
process  followed  so  far,  the  results  of  Group  A's  work,  and 
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the  importance  of  input  from  organizational  levels  below  the 
level  of  the  target  organization  level.  In  addition,  the 
briefing  to  Group  B  should  provide  an  introduction  to  pro¬ 
ductivity  measurement,  d  ^wing  on  some  specific  examples  to 
illustrate  the  concepts  efficiency  and  effectiveness. 

This  productivity  measurement  briefing  is  intended  to 
structure  the  group  activity  and  to  steer  it  toward  the 
goal  of  defining  acceptable  indicators  for  KRAs. 

Generate  productivity  indices  for  key  results  areas  -  At  the 
beginning  of  the  structured  group  process,  time  should  be 
devoted  to  a  discussion  of  the  KRA  development  process  com¬ 
pleted  by  Group  A.  Since  Group  B  includes  some  members  who 
were  also  in  Group  A,  one  of  these  individuals  could  be  asked 
to  present  this  briefing.  The  ground  rule  to  follow  in  the 
event  that  Group  B  does  not  accept  the  listing  developed  by 
Group  A  is  that  Group  B  can  add  additional  KRAs.  However, 
it  cannot  delete  any  KRA's  suggested  by  Group  A.  Once  a  set 
of  KRAs  has  been  agreed  upon,  the  procedure  moves  to  the  next 
step. 


3.3.1. 


Silent  generation.  To  begin  the  nominal  group  process, 
the  coordinator  should  select  one  of  the  KRAs  that  will 
be  relatively  easy  to  measure.  Members  of  Group  B 
should  then  be  asked  to  write  on  an  index  card  the 
indicators  which  they  feel  best  measure  the  KRA. 


3.3.2.  Round- robin  listing.  In  round  robin  fashion,  the 
coordinator  asks  members  to  share  one  of  their 
indicators.  This  process  continues  until  each  member 
has  been  asked  for  input.  The  process  is  then  repeated 
until  all  indicators  listed  by  members  have  been  shared 
and  have  been  recorded  by  the  coordinators  on  flip 
charts . 


3.3.3.  Clarification.  Time  is  then  allowed  for  questions  and 
clarification.  Based  on  this  discussion,  indicators 
may  be  modified  or,  with  the  consent  of  the  original 
presenter,  combined  with  another  item,  or  deleted. 

This  step  ends  when  the  indicators  proposed  by  all  group 
members  are  understood  by  the  total  group. 

3.3.4.  Vote  1.  Members  are  then  asked  to  vote  for  the  indicators 
they  feel  provide  the  best  measure  of  the  KRA.  In  de¬ 
ciding  how  to  vote,  members  are  asked  to  select  the  in¬ 
dicator  that  should  be  used  if  there  could  be  only 

one  indicator  for  the  KRA.  Then  they  select  the  next 
best  and  so  on  until  the  appropriate  number  to  be  ranked 
is  completed.  The  coordinator  determines  how  many  in¬ 
dicators  each  participant  should  be  asked  to  rank 
order  using  the  following  rule  of  thumb.  If  there  are 
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five  or  more,  then  half  should  be  rank  ordered.  For 
example,  if  the  group  lists  seven  indicators  for  a  KRA, 
then  each  member  should  be  asked  to  vote  for  4.  Members 
vote  by  listing  the  indicator  on  an  index  card  and 
assigning  a  weight.  The  "best"  indicator  gets  a  weight 
equal  to  the  number  of  items  voted  on.  The  next  best 
gets  a  vote  of  1  less  and  so  on. 

3.3.5.  Voting  and  tallying.  Members  are  then  instructed  to  go 
to  the  next  KRA  and  list  indicators.  While  they  are 
listing  indicators,  the  coordinators  are  tallying  the 
votes  for  KRA  #1.  Once  members  have  listed  indicators 
for  a  KRA,  steps  3.3.2  -  3.3.4  are  repeated.  This 
process  continues  until  indicators  have  been  developed 
for  all  KRAs.  Based  on  experience  with  this  process, 

2  sessions  of  2  to  2\  hours  each  will  be  required. 

These  sessions  should  be  held  on  consecutive  days  if 
possible  to  maintain  the  continuity  of  the  process. 

3 . 4  Develop  chart  listing  Key  Results  Areas  and  indicators  - 
Using  the  results  of  Group  B's  activity,  coordinators  should 
construct  a  chart  depicting  the  indicators  proposed  to 
measure  each  KRA. 

3.5  Refine  indicators  and  develop  data  sources.  The  next  stage 
of  the  process  involves  reviewing  the  indicators  and  defining 
the  required  sources  of  data.  The  information  is  derived  from 
Group  C  which  is  comprised  of  selected  members  of  Groups  A 
and  B  supplemented  by  individuals  who  are  familiar  with  exist¬ 
ing  data  sources.  The  coordinators  will  moderate  Group  C's 
activities  which  will  follow  a  discussion  mode.  The  chart 
prepared  in  3.4  is  the  primary  input  to  the  process.  The  co¬ 
ordinators  explain  that  the  purpose  of  this  session  is  to  review 
the  KRA's  and  indicators,  to  resolve  remaining  discrepancies 
between  Groups  A  and  B,  to  refine  the  indicators  and  to  identify 
sources  of  data  for  the  indicators.  While  availability  of  data 
has  not  been  a  significant  concern  up  to  this  point,  questions 
of  cost  effectiveness  of  data  gathering  are  important  considera¬ 
tions  for  Group  C.  For  example,  it  is  expected  that  if  two 
indicators  have  been  ranked  about  equal  in  value,  the  one  which 
involves  the  lowest  data  collection  cost  will  be  selected. 
Tradeoffs  between  the  value  of  an  indicator  and  its  cost  for 
data  collection  will  be  weighed  carefully  as  Group  C  reviews 
the  porposed  indicators.  In  qeneral ,  Group  C  will  attempt  to 
select  the  indicators  for  which  data  is  already  beinq  qathered, 
either  by  the  unit  itself  or  by  a  parent  orqani zat ion.  When 
new  data  gathering  is  necessary,  the  group  will  develop  a 
justification  in  light  of  the  alternatives  available  (e.q., 
subst i tuting  a  less  desirable  but  acceptable  measure  for  which 
data  are  available  versus  keepinq  logs  to  generate  new  data)  . 

Only  when  clearly  justified,  should  new  data  be  gathered.  If  new 
data  are  required,  the  group  will  recommend  procedures  to  collect 
it . 
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3.6  Develop  chart  of  KRA's,  indicators  and  data  sources.  The 
chart  developed  in  3.4  will  be  revised  to  reflect  the 
discussion  by  Group  C. 

3 . 7  Briefing  of  recommendations  to  target  and  parent  unit 

commanders .  Since  acceptance  of  the  recommendations  by 
commanders  is  necessary,  briefings  of  the  results  are  required. 

The  briefings  will  be  conducted  by  members  of  Group  C  and  the 
measurement  coordinators.  Normally,  the  target  unit  commander 
will  be  a  part  of  Group  C.  If  so,  the  briefing  need  only  be 
scheduled  for  the  parent  unit(s)  as  appropriate. 

Phase  4.  Generalization  to  Other  Units  (Optional) 

For  some  measurement  purposes,  it  will  be  necessary  to  have  a 
set  of  indicators  which  can  be  applied  across  similar  organizations. 

These  kinds  of  indicators  will  be  useful  for  research  projects.  Such 
measures  are  also  necessary  to  productivity  improvement  efforts  which 
generate  comparative  feedback  to  commanders  regarding  where  their 
organization's  productivity  stands  relative  to  other  similar  branches, 
squadrons,  etc. 

The  procedures  for  developing  generalizable  indicators  are  described 
below. 

4.1  Develop  measurement  plan_  Prior  to  developing  indicators  as 
described  in  Phases  1  to  3,  it  is  necessary  to  develop  a 
measurement  plan.  Development  of  such  a  plan  requires 
interaction  with  different  headquarters  depending  on  the 
scope  of  the  effort.  The  purpose  of  the  contacts  is  to 
obtain  clearance,  cooperation,  and  information  regarding 
potential  target  organizations  to  serve  as  measurement 
locations.  To  facilitate  this  process  Air  Force  Functional 
Account  Codes  for  the  organizations  might  be  used  to  identify 
the  population  of  organizations. 

4.2  Select  sample  measurement  locations.  From  the  population  of 
organizations  identified  by  Functional  Account  Codes,  a 
sample  will  be  chosen  to  serve  as  actual  measurement  locations. 
Table  8  provides  guidelines  for  determininq  the  size  of  this  sa:iple. 

If  all  work  centers  fall  within  the  same  MAJCOM,  then  simple 
random  sampling  can  be  used  to  select  work  centers.  If 
multiple  commands  are  involved,  the  number  of  work  centers 
randomly  selected  from  a  given  MAJCOM  should  bear  the  same 
percentage  relationship  to  the  total  sample  size  as  the  number 
of  work  centers  in  the  given  MAJCOM  bears  to  the  total  number 
of  work  centers  in  the  Air  Force. 
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then  the  minimum  number  to 
be  measured  will  be: 


If  the  total  number  of  work 
center  locations  is: 


1-3 

all 

4-7 

4 

8-13 

5 

14-20 

6 

21-30 

7 

31-44 

8 

45-65 

9 

66-100 

10 

101-more 

10%  of  total 

aFrom  AFR  25-5  Vol.  II,  November  4,  1977,  Table  3-3. 


I 

i 


I 


t 


j 


,1 


i 


4.3  Develop  plan  for  field  work.  Once  the  sample  is  developed,  a 
plan  should  be  developed  that  will  outline  the  approach  to 
obtaining  cooperation,  the  schedule  for  field  visits  and  the 
procedures  to  be  followed  within  each  measurement  location. 

The  plan  should  also  indicate  the  necessary  support  requirements 
required  from  participating  organizations. 

4.4  Obtain  clearance  from  appropriate  headquarters  .  Once  the  plan 
has  been  developed,  it  should  be  submitted  or  briefed  to  the 
appropriate  headquarters  (AF  or  MAJCOM)  for  approval . 

4.5  Finalize  plans  for  field  visits  to  measurement  locations. 

With  approval  of  the  plan.  Headquarters  will  designate  a 
point  of  contact  for  each  measurement  location  and  the 
Headquarters.  Working  jointly  with  individuals  at  both 
contact  points,  a  final  schedule  of  visits  should  be  establish¬ 
ed. 

4.6  Implementation  of  the  methodology. 

4.6.1.  Measurement  location  1.  At  measurement  location  1  the 
full  procedure  would  be  implemented  as  described  for 
Phases  1  to  3  above.  The  result  would  be  the  chart 
described  in  section  3.6. 

4.6.2.  Measurement  location  2.  Procedures  carried  out  at 
measurement  location  2  would  be  designed  to  confirm  or 
expand  the  results  from  measurement  location  1.  In 
addition,  any  KRAs  and  indicators  which  should  be 
added  for  organization  2  but  were  not  generated  for 
location  1  would  be  added.  Generally,  this  would  be 
accomplished  in  an  abbreviated  application  of  the 
methodology.  Groups  A  and  B  would  be  formed  as 
previously  described,  however,  unless  new  KRAs  or 
indicators  are  added,  the  nominal  group  procedures  would 
not  be  utilized. 

During  the  orientation  processes  for  Groups  A  and  B, 
each  would  be  given  the  results  from  the  comparable 
group  obtained  in  organization  1.  Each  group  would 
be  asked  to  review  the  list  of  KRAs  (Group  A)  and 
indicators  (Group  B)  obtained  from  location  1  and 
indicate  whether  they  are  relevant  for  organization  2. 
Next,  if  Group  A  in  the  second  organization  adds  KRAs, 
then  Group  B  would  be  asked  to  generate  indicators  for 
the  added  KRAs.  The  nominal  group  procedures  would  be 
used  in  this  case  to  generate  new  KRAs  and  indicators. 

Group  C  would  be  convened  to  consider  the  resulting 
KRAs  and  indicators  developed  from  location  2.  This 
discussion  would  be  conducted  as  discussed  in  paragraph 
3.5. 
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4.6.3.  Remaining  measurement  locations.  The  abbreviated  meth¬ 
odology  would  be  repeated  at  all  remaining  locations. 

The  additional  KRAs  and  indicators  as  well  as  variations 
in  data  sources,  would  be  carefully  noted  for  each  new 
location. 

4.6.4.  Develop  common  set  of  KRAs.  indicators,  and  data  sources. 
Following  visits  to  all  measurement  locations  in  the 
sample,  the  measurement  coordinators  will  reconcile 

the  differences  and  define  a  set  of  common  KRAs  and 
indicators  which  are  consistent  and  meaningful  across 
all  measurement  locations  in  the  sample.  This  result¬ 
ing  product  would  be  submitted  to  participating  organiza¬ 
tions  for  review  and  comment. 

4.6.5.  Develop  summary  chart.  Measurement  coordinators  will 
develop  a  chart  summarizing  this  resulting  common  set 
of  KRAs  and  productivity  indicators.  They  will  also 
note  the  data  sources  and  any  variations  among  measure¬ 
ment  locations  that  must  be  taken  into  consideration. 

Phase  5.  Implementation 

Once  the  final  set  of  KRAs  and  indicators  are  established  either  for 
a  single  organization  by  applying  Phases  1  to  3,  or  for  multiple  organiza¬ 
tions  by  applying  Phases  1  to  4,  the  next  step  is  to  implement  the 
measurement  process.  The  implementation  phase  consists  of  collecting 
the  necessary  data  to  compute  the  productivity  indicators  within  each 
KRA.  The  implementation  phase  will  vary  as  a  function  of  the  purposes 
for  which  measurement  is  undertaken  and  characteristics  of  the  organiza¬ 
tions  involved.  Therefore,  a  detailed  description  of  a  field  implementa¬ 
tion  of  the  methodology  is  beyond  the  scope  of  the  present  report. 
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Vlt.  APPLICATION  OF  THE  METHODOLOGY:  AN  EXAMPLE 


This  section  describes  a  field  demonstration  of  the  productivity 
measurement  methodology  conducted  in  an  Air  Force  maintenance  organiza¬ 
tion.  The  purposes  were  to  demonstrate  the  feasibility  of  the  approach 
in  an  Air  Force  environment  and  to  illustrate  the  nature  of  the  results 
which  might  be  obtained  by  applying  the  methodology  to  an  Air  Force 
organization.  Since  only  a  single  organization  was  involved  in  the 
demonstration,  Phase  4  of  the  methodology,  which  describes  how  the 
criteria  can  be  generalized  across  organizational  settings,  was  not 
utilized.  Description  of  the  example  will  follow  the  phases  and  steps 
used  to  describe  the  methodology  in  the  preceeding  section.  Because 
this  was  a  simulation,  it  did  not  make  use  of  all  steps  in  the  methodology 
described  in  the  preceeding  chapter. 

Phase  1.  Background 
1.1  Decision  to  measure  productivity 

For  the  purpose  of  the  illustration,  the  reason  for  measuring  pro¬ 
ductivity  was  assumed  to  be  the  development  of  baseline  measurement 
indices  for  the  target  organization  that  could  be  used  to  assess  the 
impact  of  a  hypothetical  organizatonal  change  program.  To  obtain  access 
to  an  organization  to  serve  as  the  demonstration  site,  contact  was  made 
with  the  appropriate  MAJCOM  headquarters  through  written  correspondence  with 
the  Command  productivity  principal.  Following  this  discussion,  a  squadron 
level  organization  was  selected  with  the  understanding  that  the  measure¬ 
ment  coordinator,  in  conjunction  with  the  squadron  commander,  would  select 
a  branch  to  be  the  target  organization. 

In  an  operational  situation,  the  decision  to  measure  productivity 
would  normally  be  made  by  the  unit  commander.  This  decision  might  be 
motivated  by  a  desire  to  improve  the  unit's  performance  or  to  comply 
with  a  directive  from  higher  headquarters  (e.g. ,  compliance  with  MAJCOM 
or  Air  Force  productivity  plan) . 

1.2.  Select  Measurement  Coordinator 

The  author  served  as  the  measurement  coordinator  for  this  field 
demonstration . 

1.3.  Familiarization  with  the  Organization 

The  process  of  organizational  familiarization  started  with  an 
orientation  meeting  between  the  measurement  coordinator  and  the  base 
productivity  principal.  Next,  a  meeting  was  held  with  the  wing  maintenance 
officer,  the  squadron  commander,  officers  and  non-commissioned  officers 
(NCOs)  from  the  squadron  headquarters,  and  representatives  of  each  work 
center.  The  target  squadron  was  an  avionics  maintenance  squadron.  In 
This  initial  meeting,  the  measurement  coordinator  explained  the  purpose 
of  the  measurement  effort  and  the  general  procedures  to  be  followed. 

Time  was  allowed  for  squadron  representatives  to  raise  questions  about  the 
process . 


This  introductory  meeting  was  followed  with  a  short  meeting  with  the 
squadron  commander  in  which  the  methodology  was  discussed  in  more  detail. 
In  turn,  the  squadron  commander  provided  an  overview  of  the  squadron 
organization  and  mission.  Based  on  criteria  established  by  the  measure¬ 
ment  coordinator,  the  Communications/Navigation  branch  was  selected  as 
the  target  branch.  This  meeting  concluded  with  the  squadron  commander 
conducting  a  brief  tour  of  the  squadron's  activities,  and  helped  the 
measurement  coordinator  visualize  the  organization  and  operation  of  the 
squadron.  Having  the  squadron  commander  introduce  the  coordinator  to 
squadron  members  also  paved  the  way  for  the  excellent  cooperation 
received.  Finally,  the  squadron  commander  provided  a  document  which 
described  the  major  subfunctions  within  the  squadron. 

Background  discussions  were  also  held  with  the  squadron  maintenance 
officer  who  provided  copies  of  the  "Maintenance  Digest,"  a  monthly 
report  of  the  deputy  chief  of  staff  maintenance.  This  document  lists 
maintenance  indicators  that  are  routinely  tracked  at  the  wing  level. 

Also  provided  was  a  document  summarizing  the  results  of  recent  quality 
control  inspections  which  listed  the  number  and  type  of  deficiencies 
found  for  each  work  center. 

The  final  orientation  interview  was  conducted  with  the  branch 
Officer  in  Charge  (OIC)  to  obtain  information  concerning  the  organiza¬ 
tion  of  the  branch.  This  discussion  focused  on  the  mission,  inputs  and 
outputs  and  other  organizations  that  the  branch  must  come  into  contact 
with. 


A  similar  process  would  be  followed  in  an  operational  application 
of  the  methodology. 

1.4.  ■  Define  Organizational  Boundaries 

Since  the  Communications/Navigation  Branch  is  a  clearly  defined 
organizational  entity,  the  boundaries  are  quite  clear.  Included  in  the 
branch  organization  are  the  NCOIC,  the  OIC,  and  three  work  centers. 
Individuals  who  were  assigned  to  the  branch  but  who  were  on  detail  were 
not  considered,  for  measurement  purposes,  to  be  a  part  of  the  branch. 

1.5.  Construct  Organizational  Diagram 

Based  on  the  information  obtained  in  1.3  and  1.4,  the  diagram  shown 
Figure  8  was  prepared. 


Phase  2.  Definition  of  Key  Results  Areas 

2.1  Form  Management  Group  -  Group  A 

In  this  illustration,  the  Management  Group  consisted  of  the  follow¬ 
ing: 
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Branch  0.1 


Organizational  Diagram  —  Communications /Navigation  Branch 


r 


Squadron  Level 

Maintenance  Officer  -  Captain 
Maintenance  Superintendent  -  Civilian 

Branch  Level 

Branch  OIC  -  1/Lt. 

Branch  NCOIC  -  C/MSGT 

Section  Level 

Radio  Shop  NCOIC  -  C/MSGT 
Radar  Shop  NCOIC  -  S/MSGT 
Computer  Shop  NCOIC  -  S/MSGT 

2.2.  Orient  Group  A 

In  this  illustration,  the  orientation  had  begun  in  Phase  1  (1.3) 
during  the  initial  meeting  with  squadron  representatives.  More  specific 
discussions  of  the  purpose  and  objectives  of  the  measurement  activity 
were  held  in  individual  interviews  with  members  of  Group  A. 

2.3.  Generate  Key  Results  Areas 

The  nominal  group  process  was  not  carried  out  in  this  field  demonstra¬ 
tion.  Instead,  individual  interviews  were  held  with  members  of  Group  A 
in  which  each  member  was  asked  to  go  through  the  same  thought  processes 
that  would  have  been  required  if  the  nominal  group  procedures  had  been 
used.  Generating  KRA's  in  the  group  setting  is  probably  less  demanding 
on  the  part  of  participants  than  the  individual  interview  process.  There¬ 
fore,  since  the  interviews  produced  useful  results,  it  is  reasonable  to 
infer  that  the  nominal  group  procedures  used  in  an  operational  application 
of  the  methodology  would  have  also  yielded  useful  indicators. 

2.4  Develop  Chart  Listing  and  Describing  Key  Results  Areas 

As  a  result  of  the  individual  interviews  with  group  members  a  number 
of  Key  Results  Areas  were  identified.  These  are  listed  in  Table  9. 

Because  the  gro-up  activity  was  not  carried  out,  these  were  not  prioritized. 


Phase  3.  Definition  of  Productivity  Indices  and  Data  Sources 
3 . 1  Form  Group  B 

In  the  present  example.  Group  B  was  comprised  of  the  following: 
Radio  Shop  NCOIC  -  C/MSGT 
Computer  Shop  NCOIC  -  S/MSGT 
Radar  Shop  NCOIC  -  S/MSGT 

Radio  Shop  Technicians  (one  5-level  and  one  7-level  military 

and  one  7-level  civilian) 
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Table  9 

Key  Results  Areas  from  Group  A 


1.  Flight  Schedule  Reliability  -  The  extent  to  which  scheduled  missions 

are  delayed  or  aborted  due  to  avionics  related  problems. 

2.  Component  Repair  Efficiency  -  The  extent  to  which  work  is  performed 

in  accordance  with  estimates  and  the  timeliness  in  which  work  is 
performed. 

3.  Component  Repair  Quality  -  The  extent  to  which  repair  work  is  per¬ 

formed  without  errors  or  discrepancies  and  in  accordance  with 
specified  procedures. 

4.  Economy  of  Repair  -  The  extent  to  which  work  is  performed  correctly 

with  a  minimum  of  new  parts  purchased. 

5.  Morale  of  the  Unit  -  The  extent  to  which  individuals  take  pride  in 

themselves  and  the  unit,  attempt  to  improve  their  physical  and 
social  environment,  seek  opportunities  for  self-improvement  and 
to  participate  in  group  activities. 

6.  Training  -  The  extent  to  which  individuals  are  encouraged  and  assisted 

to  upgrade  their  technical  skills.  The  extent  to  which  individuals 
are  current  on  Air  Force  required  training. 

7.  Demonstrated  Management  Concern  for  People  -  The  extent  to  which 

management  of  the  unit  takes  actions  to  promote  the  development  of 
individuals  in  the  organization. 

8 .  Safety  -  The  extent  to  which  work  is  performed  in  accordance  with 

prescribed  safety  practices,  the  proper  use  and  care  of  safety 
equipment,  and  the  avoidance  of  lost  time  accidents  either  at  work 
or  during  off-duty  hours. 

9.  Maintenance  of  Tools  and  Test  Equipment  -  The  extent  to  which  shop 

tools  and  test  equipment  are  properly  maintained  and  secured. 

10.  Maintenance  Data  Collection  Reliability  -  The  extent  to  which  entries 

to  the  maintenance  data  collection  system  are  made  accurately, 
completely  and  on-time. 

11.  Innovation  -  The  extent  to  which  the  organization  initiates  needed 

changes  to  the  technical  data  system. 

12.  Readiness  for  Mobilization  -  The  extent  to  which  the  organization 

maintains  a  high  degree  of  readiness  for  performing  its  war-time 
mission. 
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Table  9 


(Con't) 


Key  Results  Areas  from  Group  A 

13 .  Unit  Self  Sufficiency  -  The  extent  to  which  avionic  component  repairs 

can  be  made  at  the  local  level  as  opposed  to  submitting  them  to 
depot-level  maintenance. 

14.  Adaptability  -  The  extent  to  which  the  unit  can  adapt  to  new  re¬ 

quirements  and  new  missions  without  sacrificing  performance 
quality. 


I 

I 

t 
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Computer  Shop  Technicians  (one  5-level  and  one  7-level  military) 

Radar  Shop  Technicians  (one  5-level  and  one  7-level  military 

and  one  7-level  civilian) 


3.2  Orient  Group  B 

This  step  was  simulated  in  the  present  illustration  with  a  group 
interview  involving  five  military  technicians,  two  civilian  technicians, 
and  the  NCOIC  of  the  Radio  Section.  The  group  was  given  a  briefing 
regarding  the  purpose  of  the  measurement  process.  The  main  concern  of 
the  group  interview  was  to  determine  if  individuals  from  the  shop  level 
could  provide  useful  input  in  the  process  of  generating  indicators  for 
key  results  areas.  With  a  very  short  introduction  and  some  guidance 
on  the  part  of  the  measurement  coordinator,  useful  responses  were 
obtained.  Of  interest  is  that  the  responses  reflected  an  in-depth 
understanding  of  the  work  performed  and  "fair"  methods  for  evaluating 
the  results  of  the  work.  In  subsequent  discussions  with  management, 
it  was  evident  that  the  technicians  provided  a  different  and  important 
perspective  from  that  provided  by  management.  For  example,  considerable 
discussion  focused  on  the  indicator  for  the  KRA  -  Flight  Schedule 
Reliability.  The  technicians  felt  that  it  was  unfair  to  include  both 
the  number  of  delays  and  the  number  of  missions  aborted  as  indicators 
of  the  unit's  performance.  The  issue  seemed  to  be  that  mission  delays 
were  often  due  to  the  fact  that  parts  needed  for  a  iob  do  not  arrive  on 
time  from  supply.  According  to  the  bookkeeping  system,  maintenance 
"buys"  the  delay,  however,  the  technicians  point  out,  correctly,  that 
they  are  not  at  fault  in  approximately  25%  of  the  delays.  This  point 
adds  validity  to  the  use  of  non-management  personnel  in  Group  B  in 
operational  applications  of  the  methodology. 

3.3  Develop  Chart  Listing  Key  Results  Areas  and  Indicators 

The  Key  Results  Areas  and  indicators  resulting  from  the  discussion 
with  Group  B  are  shown  in  Table  10, 


3.4  Review  Indicators  and  Define  Data  Sources 

Group  C  was  simulated  by  interviews  with  maintenance  management  at 
the  squadron  level.  The  interviews  involved  reviewing  the  products 
produced  by  Group  A  and  B, assessing  whether  the  indicators  appeared 
reasonable  and  feasible,  and  determining  whether  existing  data  was 
sufficient  for  the  indicators.  In  an  operational  application  of  the 
methodology,  this  process  would  have  also  included  members  from  the  branch 
and  section  level  as  well  as  support  staff  (e.g.,  data  processing).  For 
purposes  of  this  illustration,  involvement  by  these  groups  was  not  deemed 
necessary  since  there  would  be  no  implementation  of  the  results. 

The  results  of  the  Group  C  interviews  are  discussed  below. 
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Table  10 


Group  B  Results 


Additions  to  Key  Results  Areas 

1.  Health  Status  -  The  extent  to  which  members  of  the  organization 
exhibit  health  problems  that  are  diagnosed  as  job  related  or 
the  result  of  job  stress. 


Indicators 

1 .  Flight  Schedule  Reliability 

^  No.  aborts  due  to  comm,  nav.  equipment  (current  period) 
No.  aborts  due  to  comm.  nav.  equipment  (previous  period) 

2 .  Component  Repair  Efficiency 

2  i  Actual  time  in  completion  of  repair 

Estimated  time  in  completion  of  repair 

2.2  No.  items  overdue  to  supply 

2.3  Man-hour  backlog  in  the  shop 

3.  Component  Repair  Quality 

Mean  flying  time  between  failures  for  repaired  components 

(current  period) _ 

Mean  flying  time  between  failures  for  repaired  components 
(previous  period) 

Satisfactory  quality  control  reports 
Total  no.  of  quality  control  reports 

4 .  Economy  of  Repair 
(No  indicator  proposed) 

5.  Morale  of  the  Unit 

Percentage  of  personal  appointments  met  (current  period) 
Percentage  of  personal  appointments  met  (previous  period) 

Retention  rate  for  career  fields 
A.F.  average  for  career  field 

No.  requests  for  transfers  out  of  unit 
Total  personnel  assigned 


5.1 

5.2 

5.3 


3.1 


3.2 
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No.  pieces  inspected 

Percentage  of  tool  kits  with  missing  or  unservicable  tools 
(current  period) 


Percentage  of  tool  kits  with  missing  or  unservicable  tools 
(previous  period) 


Maintenance  Data  Collection  Reliability 


aper  work  entries  to  maintenance  data  system  w/o  errors 


Total  no.  entries  to  maintenance  data  system 
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Table  10  (Con't) 


Group  B  Results 


11 .  Innovation 

11.1  No  acceptable  tech,  data  changes/Total  no.  technicians 
(7-level) 

11.2  No.  accepted  suggestions/Total  no.  eligible  personnel 

12.  Readiness  for  Mobilization 

(No  indicators  generated  specifically  for  this  key  results  area 
since  "Readiness  was  considered  a  composite  of  most  of  the  other 
KRA's.) 

13 .  Unit  Self  Sufficiency 

(No  indicators  generated  by  Group  B) 

14.  Adaptability 

(No  indicators  generated  by  Group  B) 

15 .  Health  Status 

^  Percentage  of  dispensary  visits  for  stress  related  problems 

(current  period) _ _ _ 

Percentage  of  dispensary  visits  for  stress  related  problems 
(previous  period) 


APR  is  Airman  Performance  Report 
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3.5  Develop  Chart  Showing  Key  Results  Areas,  Indicators  and  Data  Sources 

Table  11  summarizes  the  final  results  from  the  Group  C  process. 

A  comparison  of  Table  11  with  Tables  9  and  10  illustrates  the  changes 
in  Key  Results  Areas  and  indicators  which  took  place  during  application 
of  the  methodology. 

The  original  list  of  Key  Results  Areas  generated  by  Group  A  (Table  9) 
was  modified  during  the  work  with  Groups  B  and  C.  Group  B  added  a 
Key  Results  Area  which  was  labeled  "Health  Status."  Group  C  deleted 
two  Key  Results  Areas  which  were  included  in  the  original  list  (Tabic- 9  ): 
Adaptability  and  Readiness  for  Mobilization.  Adaptability  was  deleted 
because  no  indicators  were  generated  for  it.  Readiness  for  Mobilization 
was  deleted  because  it  was  viewed  by  management  as  a  composite  of  the  other 
areas. 

Group  C  also  made  some  changes  in  the  indicators.  Some  indicators 
were  deleted  and  others  were  added.  For  example,  under  the  KRA 
Number  11,  Innovation,  indicator  11.2  was  dropped.  Maintenance  management 
felt  that  submissions  to  the  Air  Force  Suggestion  System  were  not  a  use¬ 
ful  indicator  since  previous  technical  suggestions  submitted  by  the 
unit  were  reviewed  by  individuals  who  lacked  the  knowledge  to  appreciate 
the  suggestion's  value  to  technical  organizations. 

3.7.  Briefing  Unit  Commander 

A  final  discussion  was  held  with  the  branch  OIC  and  the  Squadron 
Commander  informing  them  of  the  results  of  the  procedure.  In  an 
operational  application  of  the  procedure,  a  formal  briefing  would  be 
held  for  the  Unit  Commander  and  higher  command  levels  as  appropriate 
following  the  Group  C  activity.  The  purpose  of  the  briefing  would  be 
to  inform  commanders  and  to  obtain  the  formal  acceptance  of  the  Key 
Results  Areas  and  indicators  developed. 

Phase  4.  Generalization  of  Indicators 

Since  this  example  was  not  concerned  with  developing  indicators 
that  could  be  generalized  across  organizations,  Phase  4  was  not  required. 

Phase  5,  Implementation 

As  a  simulated  field  test,  no  implementation  of  the  indicators 
was  intended.  However,  it  is  clear  from  the  results  obtained  that  the 
measures  are,  with  one  or  two  exceptions,  implementable  within  reason¬ 
able  cost  constraints.  Many  of  the  proposed  indicators  are  currently 
being  "tracked"  informally.  There  was  a  general  consensus  among  squadron 
management  that  one  contribution  of  the  methodology  was  to  force  them  to 
formalize  assessment  of  organizational  outcomes  that  are  already  being 
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monitored  by  informal  "management"  processes.  The  procedure  also 
benefits  organizational  members.  By  making  explicit  the  criteria  on 
which  managers  make  judgements,  it  promotes  improved  communication 
among  managers,  supervisors  and  members  with  respect  to  the  important 
dimensions  of  organizational  performance. 


Summary  and  Critique 

This  chapter  describes  a  field  demonstration  of  a  methodology  re¬ 
commended  for  use  in  generating  productivity  criteria  in  Air  Force 
organizations.  The  demonstration  was  conducted  in  an  avionics  maintenance 
squadron  within  the  Military  Airlift  Command.  This  demonstration  was 
intended  as  an  illustration,  rather  than  a  test  of  the  methodology. 
Nevertheless,  the  demonstration  provided  some  support  for  the  feasibility 
of  applying  Phases  1  to  3  of  the  methodology  in  an  Air  Force  organization. 
Further  evaluation  of  the  methodology  will  await  the  outcome  of  an 
extensive  field  test  to  be  conducted  under  the  present  contract. 

Based  on  the  results  of  the  field  demonstration,  there  is  preliminary 
evidence  that  the  methodology  will  provide  criteria  which  satisfy  the 
requirements  specified  for  desirable  productivity  measures  in  Chapter  IV 
of  this  report.  Compared  to  existing  productivity  measurement  programs 
underway  in  the  Air  Force,  this  methodology  appears  to  have  more  potential 
benefits  in  terms  of  providing  meaningful  feedback  to  local  commanders. 

The  measures  resulting  from  this  methodology  are  congruent  with  the 
mission  of  the  organization,  and  they  are  measurable,  frequently  through 
the  use  of  existing  data.  Acceptability  to  local  commanders  and  organiza¬ 
tional  members  is  enhanced  by  their  participation  in  the  process.  This 
participation  leads  to  a  set  of  measures  which  are  choosable  and  under¬ 
standable  from  the  point  of  view  of  the  organization  in  question. 

Questions  regarding  the  validity,  cost-effectiveness,  and  useful¬ 
ness  of  the  measures  to  researchers  and  managers  will  be  answered  in 
part  by  the  planned  field  test.  However,  further  research  in  the 
operational  environment,  if  judged  desirable  based  on  the  results  of  the 
field  test,  will  ultimately  be  required  to  answer  these  questions.  A 
major  benefit  to  the  Air  Force  from  this  line  of  research,  will  be  the 
ability  to  evaluate  the  impact  of  management  and  policy  changes  on  both 
operational  efficiency  and  effectiveness.  Objective  criteria  of  both 
efficiency  and  effectiveness  are  required  in  order  to  evaluate  trade¬ 
offs  between  efforts  to  promote  peacetime  efficiency  and  maintenance  of 
operational  readiness.  The  ability  to  evaluate  such  tradeoffs  is  a  primary 
concern  of  both  Air  Force  managers  and  researchers,  and  is  a  planned  out¬ 
come  of  this  present  line  of  productivity  measurement  research. 
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APPENDIX  A 

PROGRAMS  TO  ENHANCE  PRODUCTIVITY:  REVIEW  OF  RESEARCH  RESULTS 


The  primary  purpose  of  this  report  is  to  review  and  critique 
productivity  measurement  methods.  However,  since  the  ultimate  purpose 
of  measurement  is  to  enhance  productivity,  it  is  appropriate  to  review 
the  highlights  of  the  productivity  enhancement  literature.  This  is  the 
purpose  of  this  Appendix. 

In  recent  years,  there  have  been  at  least  four  significant  reviews 
of  the  literature  pertaining  to  the  effectiveness  of  behavioral  and 
management  strategies  for  improving  productivity  (Cummings  &  Molloy,  1977; 
Katzell,  Yankelovitch,  Fein,  Ornati,  &  Nash,  1975;  Locke,  Feren,  McCaleb, 
Shaw,  &  Denny,  1979;  Spector  &  Hayes,  1979).  Each  has  reviewed  a 
substantial  number  of  research  studies  and  has  categorized  the  impact  of 
various  improvement  programs  on  a  range  of  criteria  including  productivity 
withdrawal,  disruptions  and  employee  attitudes.  While  the  literature 
covered  by  these  reviews  overlapped  (to  an  unknown  degree) ,  the  total 
number  of  citations  reviewed,  including  duplicates  was  approximately  290. 
This  Appendix  provides  a  summary  of  the  results  of  these  reviews  organized 
according  to  the  major  types  of  improvement  programs. 

A  classification  scheme  proposed  by  Katzell,  Bienstock,  and  Faerstein 
(1977)  will  be  used  to  organize  this  discussion  of  enhancement  methods. 
This  scheme  considered  enhancement  programs  under  the  following  categories 

1 .  Selection  and  Placement 

2.  Job  Development  and  Promotion 

3.  Training  and  Instruction 

4.  Appraisal  and  Feedback 

5.  Management  by  Objectives 

6.  Goal  Setting 

7.  Financial  Compensation 

8.  Job  Design 

9.  Group  Design 

10.  Supervisory  Methods 

11.  Organizational  Structure 

12.  Physical  Working  Conditions 

13.  Work  Schedule 

14.  Socio-technical  Systems 

Assigning  the  studies  included  in  the  four  reviews  to  the  14 
categories  presented  some  difficulty.  The  categories  represent  the 
"independent  variables"  manipulated  in  an  effort  to  improve  labor 
productivity.  Frequently  programs  involved  manipulations  of  more  than 
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one  "action  lever"  or  variable.  For  example,  an  experiment  in  job  re¬ 
design  may  be  accompanied  or  followed  by  a  change  in  the  compensaton 
system.  When  this  was  the  case,  Katzell  et  al.,  (1975)  included  the 
study  in  each  of  the  relevant  categories.  The  other  three  reviews 
assigned  articles  to  discrete  categories  based  on  the  dominant  change 
strategy  involved.  In  this  discussion  categories  defined  in  all  four 
reviews  were  assigned  to  the  one  of  the  13  Katzell  et  al.  (1977) 
categories  which  was  judged  to  be  most  similar. 

The  four  published  reviews  focused  primarily  on  the  effects  of 
the  change  programs  in  terms  of  their  impact  on  productivity.  However, 
reflecting  the  confusion  in  the  literature  regarding  the  meaning  of 
the  term,  the  definition  of  productivity  used  was  not  precise.  Probably 
the  most  frequent  definition  found  in  the  articles  reviewed  was  the 
definition  of  productivity  as  production,  although  production  costs  and 
quality  were  also  considered.  Other  criteria, in  addition  to  productivity 
used  to  assess  the  results  of  improvement  efforts,  included  withdrawal 
(absenteeism,  turnover,  tardiness,  etc.),  disruptions  (accidents,  strikes, 
slowdowns,  grievances,  etc.)  and  employee  attitudes. 

This  Appendix  presents  the  frequency  and  percentage  with  which  the 
results  of  each  type  of  enhancement  program  reported  improvement  in 
criteria  of  productivity,  withdrawal,  disruptions  and  attitudes.  In 
interpreting  these  results,  it  should  be  kept  in  mind  that  the  percentages 
probably  provide  an  inflated  estimate  of  the  true  percentage  of  success 
since  unsuccessful  programs  rarely  get  published.  In  any  event,  the 
numbers  provide  a  crude  estimate  of  the  level  of  research  activity  in  each 
of  the  improvement  program  areas. 

Selection  and  Placement 

Table  A-l  summarizes  the  results  of  studies  that  assessed  the 
consequences  on  productivity  of  a  change  in  how  the  organization  was 
staffed.  Staffing  refers  to  activities  associated  with  recruitment, 
selection, and  placement. 


Job  Development  and  Promotion 

This  category  was  created  to  consider  the  impact  of  career  develop¬ 
ment  and  promotion  systems  on  productivity.  However,  there  were  no 
expermental  evaluations  of  such  programs.  Thus,  no  table  is  presented 
for  this  category. 

Training  and  Instruction 

Results  of  a  variety  of  training  and  instruction  programs  are  presented 
in  Table  A-2.  These  included  supervisory  training  and  worker  training 
involving  a  variety  of  training  techniques.  As  a  means  of  summarizing  these 
programs,  Katzell,  et  al.  (1977)  have  found  that  training  programs  have 
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Table  A-l 


Number  and  Percentage  of  Studies  Showing  Improvement  from 
Selection  and  Placement  Programs 


CRITERION 

Review  Productivity 

Withdrawal 

Disruptions 

Attitudes 

Katzell,  et  al. (1975) 

- 

a 

1/3  (33%) 

- 

1/1  (100%) 

Cummings  &  Molloy  (1977) 

- 

- 

- 

- 

Locke,  et  al.  (1979) 

- 

- 

- 

- 

Spector  &  Hayes  (1979) 

- 

- 

- 

- 

al  out  of  3  studies  reviewed  showed  improvement  on  the  withdrawal 
criterion 
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bf?»n  used  with  success  in  a  variety  of  settings.  However,  not  all  proqrams 
tried  have  been  successful,  "...in  short,  training  may  fail  when  tried  as 
a  remedy  for  tha  wrong  disease"  (Katze)  1  ,  et  al  .  ,  1977,  n.  131. 

Table  A- 2 

Number  and  Percentage  of  Studies  Showing  Improvement  from 
Training  and  Instruction  Programs 

CRITERTON 

Review  Productivity  Withdrawal  Disruptions  Attitudes 

Katzell,  et  al.  (1975)  22/26  (85%)  7/12  (58%)  1/1  (100%)  2/3  (67%) 

Cummings  &  Molloy  (1977)  -  -  - 

Locke,  et  al .  (1979)  -  -  - 

Spector  &  Hayes  (1979)  -  -  - 


Appraisal  and  Feedback 

Table  A-3  summarizes  results  of  programs  designed  to  provide 
guidance  to  employees  regarding  their  work  performance  through  feed¬ 
back,  counseling,  or  coaching.  Most  of  the  studies  differ  from  old- 
style  performance  appraisal  programs  when  an  appraisal  interview  is 
held  every  6  to  12  months.  These  approaches  frequently  involve  what 
is  termed  "behavior  modification"  or  positive  reinforcement  procedures 
that  are  characterized  by  feedback  that  is  more  focused,  frequent  and 
specific.  Desired  behaviors  are  clearly  specified,  and  each  employee's 
performance  or  the  group’s  performance  is  frequently  assessed  and 
reported  in  relation  to  these  specific  objectives.  Feedback  is  frequently 
assessed  and  reported  in  relation  to  these  specific  objectives.  Feedback 
is  frequently  accompanied  by  praise  for  good  performance. 


Management  by  Objectives 

This  approach  has  become  well-established  in  the  field  of  management 
practice.  While  it  involves  the  features  of  appraisal  and  feedback,  the 
key  feature  is  the  definition,  in  light  of  more  comprehensive  organiza¬ 
tional  objectives,  of  performance  targets  or  "bogeys"  for  each  individual 
manager's  job.  The  number  of  experimental  evaluations  of  this  approach 


Table  A-3 


Number  and  Percentage  of  Studies  Showing  Improvement  from 
Appraisal  and  Feedback  Programs 


CRITERION 

Review  Productivity  Withdrawal  Disruptions  Attitudes 


Katzell , 

et  al.  (1975) 

5/6 

(83%) 

1/3 

(33%) 

1/1  (100%) 

2/3 

(67%) 

Cummings 

&  Molloy  (1977) 

3/3 

(100%) 

3/3 

(100%) 

- 

0/1 

(0%) 

Locke ,  et 

al.  (1979) 

- 

- 

- 

- 

Spector  & 

Hayes  (1979) 

- 

- 

- 

- 
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has  hoop  fev  and  the  results  mixed.  While  performance  improvements. 
Table  A-4,  occurred,  grievances  increased  in  one  estab1 ishment.  and  the 
performance  improvements  declined  in  another  after  6  months. 

Table  A-4 

Number  and  Percent.aae  of  Studies  Showincr  Tmorovement  ^rom 
Management  by  Objectives 


CRITERION 

Review  Productivity  Withdrawal  Disruptions  Attitudes 

Katzell ,  et  aW1975)  1/2  (50%)  1/1  (100%)  0/1  (0%)  1/1  (100%) 

Cummings  &  Molloy  (1977)  -  - 

Locke,  et  al. (1979)  - 

Spector  &  Hayes  (1979)  -  - 


Goal  Setting 


Closely  related  to  feedback  and  management  by  objectives,  goal 
setting  is  characterized  by  the  specification  of  difficult  but  attainable 
goals  for  limited — but  not  unimportant — aspects  of  employee  performance. 
Normally,  these  programs  also  involve  frequent  and  specific  feedback 
regarding  goal  attainment.  The  similarity  of  this  approach  to  the 
"positive  reinforcement"  programs  included  under  the  category  of 
Appraisal  and  Feedback  prompted  the  following  observation  by  Locke, 
et  al.:  "...behavior  modification  has  been  widely  touted, but  in 
practice  involves  little  more  than  a  re-labeling  of  one  or  more  of  the 
(other)  techniques,  especially  money  and  goal  setting"  (1979,  p.  1). 

The  results  of  goal  setting  programs  are  shown  in  Table  A-5. 

Table  A-5 

Number  and  Percentage  of  Studies  Showing  Improvement  From 

Goal  Setting 


CRITERION 

Review  Productivity 

Withdrawal 

Disruptions 

Attitudes 

Katzell,  et  alil975)  8/8  (100%) 

Cummings  &  Molloy  (1977) 

Locke,  et  al.  (1979)  17/17  (100%) 

Spector  &  Hayes  (1979) 

2/3  (67%) 

1/2  (50%) 

2/3  (67%) 
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Financial  Compensation 

This  section  includes  studies  focusing  on  the  impact  of  pay.  While 
the  linking  of  pay  to  performance  has  had  a  long  history  in  manufactur¬ 
ing  operations,  there  are  many  variations  of  incentive  programs  using 
financial  rewards.  Table  A-6  presents  an  overall  summary  of  the  impact 
of  pay.  Table  A-7  presents  more  specific  breakdowns  by  type  of  financial 
incentive  program.  The  table  presents  results  for  individual  piecework 
plans,  individual  bonus  plans  (combining  goal  setting  plus  financial 
reward  for  achieving  bonus),  and  group  incentives  (Scanlon  plans  and  non- 
Scanlon  group  plans) . 


Table  A-6 

Number  and  Percentage  of  Studies  Showing  Improvement  From 
Financial  Compensation  Programs  (All  Types  Combined) 


CRITERION 

Review  Productivity 

Withdrawal 

Disruptions 

Attitudes 

Katzell ,  et  al/1975) 

6/8  (75%^ 

3/4  (75%) 

- 

- 

Cummings  &  Molloy  (1977) 

8/8  (100%) 

1/1  (100%) 

- 

7/8  (88%) 

Locke,  et  al.  (1979) 

33/37  (89%) 

- 

- 

- 

Spector  &  Hayes (1979) 

41/46  (89%) 

- 

- 

- 

Table  A-7 

Number 

and  Percentage  of  Studies  Showing  Productivity  Improvement 
Financial  Compensation  by  Program  Type 

From 

PROGRAM 

TYPE 

_  .  Individual  Individual 

Review  „ .  .  „ 

Piecework  Bonus 

Scanlon 

Other  Group 
Incentive 

Katzell , 

et  al.  (1975) 

- 

- 

Cummings 

&  Molloy  (1977) 

- 

- 

Locke,  et  al.  (1979)  10/10  (100%)  18/20  (90%) 
Spector  &  Hayes  (1979)  18/19  (95%)  13/16  (81%) 


5/7  (71%)  4/4  (100%) 

2/2  (100%)  6/6  (100%) 


One  of  the  popular  topics  in  manpower  management  over  the  past  decade 
has  been  job  redesign  to  facilitate  worker  motivation.  While  the  principal 
motivator  for  this  effort  has  been  improvement  in  workers  quality  of 
working  life,  some  have  also  argued  that  job  redesign  can  improve  per¬ 
formance.  Job  enrichment,  modifying  jobs  to  provide  workers  more  variety, 
autonomy,  feedback,  and  challenge,  has  been  the  most  widely  used  form  of 
job  redesign.  Table  A-8  summarizes  the  results  of  job  design  efforts. 

Table  A-8 

Number  and  Percentage  of  Studies  Showing  Improvement  From 
Job  Design  Programs 


CRITERION 

Review  Productivity  Withdrawal  Disruptions  Attitudes 


Katzell,  et  al. (1975) 

16/18 

(89%) 

5/6 

(83%) 

- 

2/4  (50%) 

Cummings  &  Molloy  (1977) 

16/21 

(76%) 

6/7 

(86%) 

- 

17/23  (74%) 

Locke,  et  al.  (1979) 

11/13 

(85%) 

- 

- 

- 

Spector  &  Hayes  (1979) 

- 

- 

- 

- 

Group  Design 

This  category  differs  from  job  design  in  that  it  includes  programs 
that  deal  with  redistribution  of  tasks  among  group  members  or  members 
of  a  work  team.  Job  enrichment,  on  the  other  hand,  deals  with  changes 
in  individual  jobs.  Group  Design  results  are  summarized  in  Table  A-9 


Supervisory  Methods 

One  of  the  most  common  ways  to  impact  performance  in  work  groups 
is  to  change  the  way  supervisors  perform  their  jobs.  This  category 
cross-lists  studies  cited  among  other  categories,  and  feedback.  In 
addition,  this  category  includes  studies  that  investigate  increased 
participation  (democracy)  in  the  workplace.  These  results  are 
summarized  in  Table  A-10. 


Table  A- 9 

Number  and  Percentage  of  Studies  Showing  Improvement  From 
Group  Design  Programs 

CRITERION 


Review 


Productivity  Withdrawal  Disruptions  Attitudes 


Table  A-10 


Number  and  Percentage  of  Studies  Showing  Improvement  From  Changes 

in  Supervisory  Methods 


CRITERION 

Review  Productivity 

Withdrawal 

Disruptions 

Attitudes 

Katzell,  et  al^l975)  12/13  (92%) 

4/5  (80%) 

- 

2/2 

(100%) 

Cummings  &  Molloy  (1977)  4/7  (57%) 

4/5  (80%) 

5/5 

(100%) 

Locke,  et  al.(1979)  7/16  (44%) 

- 

- 

- 

Spector  &  Hayes  (1979) 

- 

- 

- 

Organizational  Structure 

Table  A-ll  summarizes  results  of  organizational  changes  which 
alter  existing  working  relationships.  The  types  of  changes  referred  to 
are  patterns  of  responsibility  and  authority  that  are  outside  the 
immediate  work  group.  An  example  is  a  study  that  redesigned  the  chain 
of  command  thereby  creating  new  work  roles,  and  changed  existing  roles 
in  the  management  hierarchy.  Changes  are  not  so  extensive  as  to 
create  a  new  socio-technical  work  system,  a  topic  covered  later. 

Table  A-ll 

Number  and  Percentage  of  Studies  Showing  Improvement  From 
Changes  in  Organizational  Structure 


CRITERION 

Review  Productivity  Withdrawal  Disruptions  Attitudes 

Katzell,  et  al!1975)  4/4  (100%)  1/1  (100%)  1/1  (100%) 

Cummings  &  Molloy(1977)  4/4  (100%)  2/3  (67%)  -  4/5  (80%) 

Locke,  et  al. (1979)  -  - 

Spector  S  Hayes (1979)  -  - 


Physical  Working  Conditions 


1 


In  the  past,  a  great  deal  of  attention  has  been  given  to  consequences 
of  physical  working  conditions  (e.g.,  noise,  illumination,  layout) 

However,  none  of  these  reviews  covered  experimental  evaluations  of  these 
variables. 

Work  Schedule 

Table  A-12  summarizes  results  of  changes  in  work  scheduling. 

Chanqes  studied  were  compressed  work  week  (4  ten-hour  davs  rather  than 
5  eight-hour  days)  and  flexitime  (allowing  individuals  to  alter  starting 
and  stopping  time  as  long  as  they  work  a  standard  number  of  hours) . 

Table  A-12 

Number  and  Percentage  of  Studies  Showing  Improvement  From 
Changes  in  Work  Schedule 


CRITERION 

Review  Productivity  Withdrawal  Disruptions  Attitudes 


Katzell,  et  al.  (1975) 

6/9 

(67%) 

6/7 

(86%) 

- 

2/3 

(67%) 

Cummings  &  Molloy  (1977) 

5/5 

(100%) 

4/4 

(100%) 

- 

6/6 

(100%) 

Locke,  et  al.  (1979) 

- 

- 

- 

- 

Spector  &  Hayes  (1979) 

- 

- 

- 

- 

Socio-Technical  System 

This  category  includes  studies  reporting  changes  in  a  sufficiently 
large  number  of  dimensions  of  the  organization  so  as  to  create  a  new 
socio-technical  work  system.  Examples  of  the  changes  made  include  most 
of  the  following:  redesign  of  jobs  and  teams,  wider  sharing  of  respon¬ 
sibility  influence  and  authority,  wide  sharing  of  information  about 
goals,  problems,  and  results,  improvements  in  operation,  personnel 
selection, and  training.  Results  of  socio-technical  changes  are 
presented  in  Table  A-13. 
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Table  A-13 

Number  and  Percentage  of  Studies  Showing  Improvements  From 
Changes  in  Socio-Technical  Systems 


CRITERION 

Review  productivity  Withdrawal  Disruptions  Attitudes 


Katzell,  et  al. (1975) 

11/12 

(92%) 

8/10  (80%) 

1/1  (100%) 

3/3 

(100%) 

Cummings  &  Mollo.y  (1977) 

14/15 

(93%) 

5/7  (71%) 

- 

9/10 

(90%) 

Locke,  et  al.  (1979) 

- 

- 

- 

- 

Spector  &  Hayes  (1979) 

- 

- 

- 

- 

